Commit Graph

784 Commits

Author SHA1 Message Date
Alexander Tokmakov
c1c7cf56bd
Revert "[feature]: allow to attach parts from a different disk" 2024-04-11 13:43:51 +02:00
Igor Nikonov
4c106b2d0c Merge remote-tracking branch 'origin/master' into add-part-name-in-check-part-exception 2024-04-09 11:41:12 +00:00
Alexander Tokmakov
a73d927990
Merge pull request #61882 from ClickHouse/less_unformatted_exceptions
Fix 00002_log_and_exception_messages_formatting
2024-04-08 14:44:32 +00:00
Igor Nikonov
8ac9cbd80d Add part name to check part exception message 2024-04-08 14:13:43 +00:00
Azat Khuzhin
6e413223c2 Use DETACHED_DIR_NAME everywhere
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-04-05 17:49:38 +02:00
Robert Schulze
de2a0be025
Don't access static members through instance
- clang-tidy rightfully complains (-readability-static-accessed-through-instance)
- not going to enable the warning for now to avoid breaking the build
2024-04-03 18:50:33 +00:00
Arthur Passos
2ed53e309b
Reset part level upon attach from disk on MergeTree (#61536)
* hackish change to check how CI reacts

* remove unnecessary renameTo

* introduce separate metho

* remove unused arg

* rename meth od

* add simple test

* add current database

* set mutations sync

* Update tests/queries/0_stateless/03013_test_part_level_is_reset_attach_from_disk_mt.sql

---------

Co-authored-by: János Benjamin Antal <antaljanosbenjamin@users.noreply.github.com>
Co-authored-by: János Benjamin Antal <benjamin.antal@clickhouse.com>
2024-03-26 11:44:55 +01:00
Alexander Tokmakov
071a8ff95f less unformatted exceptions 2024-03-25 18:55:46 +01:00
kssenii
e3d2fb28cf Merge remote-tracking branch 'origin/master' into revert-refert-of-broken-projections-handling 2024-03-25 13:54:39 +01:00
Alexey Milovidov
9210712f4f
Merge pull request #60112 from Unalian/feat-59377
[feature]: allow to attach parts from a different disk
2024-03-24 23:12:43 +03:00
unashi
adc964568c [update] Merge the on same disk and on another disk scenarios into cloneAndLoadDataPart; remove the try catch; instead of obtaining the copied destination hard disk through traversal, the destination hard disk is obtained through reservation. 2024-03-21 15:44:27 +08:00
János Benjamin Antal
9cd7e6155c
Merge pull request #60031 from azat/merge_tree_mutations_cleanup-fix
Fix finished_mutations_to_keep=0 for MergeTree (as docs says 0 is to keep everything)
2024-03-19 12:55:46 +01:00
unashi
c78224c2ff Merge branch 'master' into feat-59377 2024-03-13 11:24:00 +08:00
Azat Khuzhin
6c74ab18c0 Do not clean mutations for MergeTree on DROP PART (to match ReplicatedMergeTree)
Even if this should be done, only related mutations should be cleaned,
not all.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-03-12 10:39:41 +01:00
Azat Khuzhin
f559767d22 Fix finished_mutations_to_keep=0 for MergeTree (as docs says 0 is to keep everything)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-03-12 10:39:41 +01:00
unashi
10c7ea7a29 [debug] fast test again 2024-03-12 14:32:07 +08:00
Zhuo Qiu
c628eaca8b Consider deleted rows when selecting parts to merge 2024-03-12 13:07:39 +08:00
Alexey Milovidov
d337379385 Remove in-memory data parts, step 2 2024-03-10 06:44:26 +01:00
Alexey Milovidov
b07e5d9f5a
Update StorageMergeTree.cpp 2024-03-10 08:11:59 +03:00
kssenii
7ac453ab4c Revert "Merge pull request #60436 from nickitat/revert_56864"
This reverts commit 8719a601fa, reversing
changes made to 657857f982.
2024-02-27 23:06:14 +08:00
Nikita Taranov
854f1b064b Revert "Merge pull request #56864 from ClickHouse/broken-projections-better-handling"
This reverts commit caaafbcedb, reversing
changes made to d45b248f96.
2024-02-27 00:13:28 +01:00
Alexander Tokmakov
ef3b19167f
Merge pull request #58036 from MikhailBurdukov/backoff_for_failed_mutations
Backoff policy for failed mutation.
2024-02-23 15:27:41 +01:00
Alexander Tokmakov
32607cc796
Merge pull request #60159 from ClickHouse/fix_create_replica
Skip sanity checks on secondary CREATE query
2024-02-23 13:53:47 +01:00
Azat Khuzhin
a4f765cae7
Improve performance of SELECTs with active mutations (#59531)
* Configure keeper for perf tests

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Improve performance of SELECTs with active mutations

getAlterMutationCommandsForPart() can be a hot path for query execution
when there are pending mutations.

- LOG_TEST - it is not only check one bool, but actually a bunch of
  atomics as well.

- Return std::vector over std::map (map is not required there) - no
  changes in performance.

- Copy only RENAME_COLUMN (since only this mutation is required by
  AlterConversions).

And here are results:

run|result
-|-
SELECT w/o ALTER|queries: 1565, QPS: 355.259, RPS: 355.259
SELECT w/ ALTER unpatched|queries: 2099, QPS: 220.623, RPS: 220.623
SELECT w/ ALTER and w/o LOG_TEST|queries: 2730, QPS: 235.859, RPS: 235.859
SELECT w/ ALTER and w/o LOG_TEST and w/ RENAME_COLUMN only|queries: 2995, QPS: 290.982, RPS: 290.982

But there are still room for improvements, at least MergeTree engines
could implement getStorageSnapshotForQuery().

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Add AlterConversions::supportsMutationCommandType(), flatten vector<vector<MutationCommand>>

* Work around what appears to be a clang static analysis bug

---------

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Co-authored-by: Michael Kolupaev <michael.kolupaev@clickhouse.com>
2024-02-22 08:51:10 +00:00
Alexander Tokmakov
ec0986af0b skip sanity checks on secondary create query 2024-02-20 21:59:28 +01:00
unashi
6437877a71 [fix] add changelog; change some feature logic 2024-02-20 20:31:59 +08:00
unashi
8de4a9dbfd [fix] delete trailing whitespaces 2024-02-20 11:42:40 +08:00
unashi
f62ccc75ec [merge] solve 2024-02-19 19:28:16 +08:00
MikhailBurdukov
c3ab9071fc
Merge branch 'master' into backoff_for_failed_mutations 2024-02-18 17:48:11 +03:00
MikhailBurdukov
b227e5dd3f Review changes 2024-02-18 13:37:12 +00:00
unashi
1549725edd [feature]: allow to attach parts from a different disk 2024-02-18 19:26:12 +08:00
Nikolai Kochetov
666b3d6664
Merge branch 'master' into allow-parallel-replicas-for-join-with-analyzer-2 2024-02-13 11:11:03 +01:00
Alexander Tokmakov
77e4b0c8d9 Revert "Merge pull request #59122 from arthurpassos/bring_back_attach_partition_pr"
This reverts commit ac90637170, reversing
changes made to fe2b5cde05.
2024-02-07 02:52:43 +01:00
Nikolai Kochetov
6b06fcf7fc Fix more tests. 2024-02-05 17:54:15 +00:00
Nikolai Kochetov
15bf263153 Support non global in mode. 2024-02-05 17:05:21 +00:00
Nikolai Kochetov
b60228af3f Allow to send a chain of subqueries for parallel replicas with analyzer. 2024-02-05 17:05:20 +00:00
MikhailBurdukov
b90a5b94a8 Move setting to mergetree 2024-02-04 11:24:26 +00:00
MikhailBurdukov
778efb8953
Merge branch 'master' into backoff_for_failed_mutations 2024-01-30 16:08:52 +03:00
MikhailBurdukov
5e845172da Enable per query setting 2024-01-30 12:42:03 +00:00
Kseniia Sumarokova
e16aaa8d81
Merge branch 'master' into broken-projections-better-handling 2024-01-30 10:54:22 +01:00
Arthur Passos
15d04d0a78
Merge branch 'master' into bring_back_attach_partition_pr 2024-01-29 15:16:49 -03:00
Nikolai Kochetov
5b2648929b
Merge pull request #59273 from kitaisreal/atomic-logger
Added AtomicLogger
2024-01-29 16:49:18 +01:00
Maksim Kita
07750ee949 Added AtomicLogger 2024-01-28 13:55:44 +03:00
Azat Khuzhin
90ab986be6 Revert "Prohibit CREATE on static storages (ATTACH should be used instead)"
CREATE is used by Replicated database even for ATTACH, and anyway ATTACH
creates format_version.txt as well (without previuos patch).

This reverts commit 323f1f320ebe7e588d443abf78efa2c60193e7a9.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-26 16:47:36 +01:00
Azat Khuzhin
66a40bda0a Revert "Prohibit DROP from static storages"
It is actually OK to DROP at least from read-only disk, as for the
write-once it should be OK as well, since CREATE is prohibited anyway,
so there should be no leftovers

CI: https://s3.amazonaws.com/clickhouse-test-reports/59170/c82050d1925439f0ede2b32acb5b1b8df4acae5d/stress_test__asan_.html
CI: https://s3.amazonaws.com/clickhouse-test-reports/59170/c82050d1925439f0ede2b32acb5b1b8df4acae5d/integration_tests__asan__[3_4].html

This reverts commit 082e62712c5e2f4c48d1abc6141ccfd1249d26b4.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-26 16:47:35 +01:00
Azat Khuzhin
aafc149c1e Prohibit CREATE on static storages (ATTACH should be used instead)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-26 16:47:35 +01:00
Azat Khuzhin
1ee723d50d Prohibit DROP from static storages
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-26 16:47:35 +01:00
Azat Khuzhin
f3b4f28f0d Disable mutable operations into {Replicated,}MergeTree with static storage
Mutable operations includes:
- INSERT
- OPTIMIZE
- parts manipulation
- ...

But does not include DROP, since you need an ability to DROP it.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-26 16:47:35 +01:00
Azat Khuzhin
6751e01cc3 Do not drop parts for static (ro/write-once) storages for Replicated engine
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-26 16:47:35 +01:00
Kseniia Sumarokova
99653fc778
Merge branch 'master' into broken-projections-better-handling 2024-01-26 16:27:23 +01:00