ClickHouse® is a real-time analytics DBMS
Go to file
Azat Khuzhin 402bf77783 Fix concurrent OPTIMIZE and DROP for ReplicatedMergeTree
Found with fuzzer [1] for 00992_system_parts_race_condition_zookeeper:

    2021.03.13 11:12:30.385188 [ 42042 ] {2d3a8e17-26be-47c1-974f-bd2c9fc7c3af} <Debug> executeQuery: (from [::1]:58192, using production parser) (comment: '/usr/share/clickhouse-test/queries/1_stateful/00153_aggregate_arena_race.sql') CREATE TABLE alter_tabl
    e (a UInt8, b Int16, c Float32, d String, e Array(UInt8), f Nullable(UUID), g Tuple(UInt8, UInt16)) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test_3.alter_table', 'r1') ORDER BY a PARTITION BY b % 10 SETTINGS old_parts_lifetime = 1, cleanup_delay_p
    eriod = 1, cleanup_delay_period_random_add = 0;
    ...
    2021.03.13 11:12:30.678387 [ 42042 ] {528cafc5-a02b-4df8-a531-a9a98e37b478} <Debug> executeQuery: (from [::1]:58192, using production parser) (comment: '/usr/share/clickhouse-test/queries/1_stateful/00153_aggregate_arena_race.sql') CREATE TABLE alter_table2 (a UInt8, b Int16, c Float32, d String, e Array(UInt8), f Nullable(UUID), g Tuple(UInt8, UInt16)) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test_3.alter_table', 'r2') ORDER BY a PARTITION BY b % 10 SETTINGS old_parts_lifetime = 1, cleanup_delay_period = 1, cleanup_delay_period_random_add = 0;
    ...
    2021.03.13 11:12:40.671994 [ 4193 ] {d96ee93c-69b0-4e89-b411-16c382ae27a8} <Debug> executeQuery: (from [::1]:59714, using production parser) (comment: '/usr/share/clickhouse-test/queries/1_stateful/00153_aggregate_arena_race.sql') OPTIMIZE TABLE alter_table FINAL
    ...
    2021.03.13 11:12:40.990174 [ 2298 ] {a80f9306-3a73-4778-a921-db53249247e3} <Debug> executeQuery: (from [::1]:59768, using production parser) (comment: '/usr/share/clickhouse-test/queries/1_stateful/00153_aggregate_arena_race.sql') DROP TABLE alter_table;
    ...
    2021.03.13 11:12:41.333054 [ 2298 ] {a80f9306-3a73-4778-a921-db53249247e3} <Debug> test_3.alter_table (d4fedaca-e0f6-4c22-9a4f-9f4d11b6b705): Removing part from filesystem 7_0_0_0
    ...
    2021.03.13 11:12:41.335380 [ 2298 ] {a80f9306-3a73-4778-a921-db53249247e3} <Debug> DatabaseCatalog: Waiting for table d4fedaca-e0f6-4c22-9a4f-9f4d11b6b705 to be finally dropped
    ...
    2021.03.13 11:12:41.781032 [ 4193 ] {d96ee93c-69b0-4e89-b411-16c382ae27a8} <Debug> test_3.alter_table (d4fedaca-e0f6-4c22-9a4f-9f4d11b6b705): Waiting for queue-0000000085 to disappear from r2 queue
    ...
    2021.03.13 11:12:41.900039 [ 371 ] {} <Trace> test_3.alter_table2 (ReplicatedMergeTreeQueue): Not executing log entry queue-0000000085 of type MERGE_PARTS for part 7_0_0_1 because part 7_0_0_0 is not ready yet (log entry for that part is being processed).
    2021.03.13 11:12:41.900213 [ 365 ] {} <Trace> test_3.alter_table2 (ReplicatedMergeTreeQueue): Cannot execute alter metadata queue-0000000056 with version 22 because another alter 21 must be executed before
    2021.03.13 11:12:41.900231 [ 13762 ] {} <Trace> test_3.alter_table2 (ae877c49-0d30-416d-9afe-27fd457d8fc4): Executing log entry to merge parts -7_0_0_0 to -7_0_0_1
    2021.03.13 11:12:41.900330 [ 13762 ] {} <Debug> test_3.alter_table2 (ae877c49-0d30-416d-9afe-27fd457d8fc4): Don't have all parts for merge -7_0_0_1; will try to fetch it instead
    ...

  [1]: https://clickhouse-test-reports.s3.yandex.net/21691/eb3710c164b991b8d4f86b1435a65f9eceb8f1f5/stress_test_(address).html#fail1
2021-03-14 22:35:50 +03:00
.github Drop truthy.check-keys from yamllint (does not supported on CI) 2021-02-21 06:15:36 +03:00
base Fix typo 2021-03-14 12:24:17 +03:00
benchmark Convert to python3 (#15007) 2020-10-02 19:54:07 +03:00
cmake Merge pull request #21597 from qoega/fixes-clang-12-cmake-3.19 2021-03-11 23:45:53 +03:00
contrib Merge branch 'master' into replace-memcpy 2021-03-13 09:39:58 +03:00
debian Auto version update to [21.4.1.1] [54449] 2021-03-06 14:43:30 +03:00
docker Update Dockerfile 2021-03-12 18:26:35 +03:00
docs Merge pull request #21391 from ka1bi4/romanzhukov-DOCSUP-7100-avg 2021-03-12 19:48:01 +03:00
programs Remove currently unused code 2021-03-13 10:38:35 +03:00
src Fix concurrent OPTIMIZE and DROP for ReplicatedMergeTree 2021-03-14 22:35:50 +03:00
tests Merge pull request #16240 from ianton-ru/s3_zero_copy_replication 2021-03-14 18:19:37 +03:00
utils Merge branch 'master' into replace-memcpy 2021-03-13 09:39:58 +03:00
website boop the ci 2021-03-13 15:43:02 +03:00
.arcignore Added .arcignore 2020-05-21 09:17:03 +03:00
.clang-format Fixed wrong code around Memory Profiler 2020-03-03 03:24:44 +03:00
.clang-tidy Enable clang-tidy for programs and utils 2020-05-18 04:19:50 +03:00
.editorconfig Changed tabs to spaces in editor configs and in style guide [#CLICKHOUSE-3]. 2017-04-01 11:35:09 +03:00
.gitattributes Union merge for arcadia_skip_list.txt to avoid frequent conflicts 2021-03-10 08:50:32 +03:00
.gitignore ignore data store files 2021-02-17 12:58:17 +08:00
.gitmodules Merge branch 'master' into in_memory_raft 2021-02-01 14:30:11 +03:00
.potato.yml Fix yamllint issues 2021-02-20 23:25:21 +03:00
.pylintrc Add pylintrc config 2021-01-26 23:35:56 +03:00
.vimrc Changed tabs to spaces in editor configs and in style guide [#CLICKHOUSE-3]. 2017-04-01 11:35:09 +03:00
.yamllint Drop truthy.check-keys from yamllint (does not supported on CI) 2021-02-21 06:15:36 +03:00
AUTHORS Update AUTHORS 2020-01-23 17:36:05 +03:00
CHANGELOG.md Update CHANGELOG.md 2021-03-12 11:41:20 +03:00
CMakeLists.txt Remove useless CMake option 2021-03-14 14:08:08 +03:00
CODE_OF_CONDUCT.md Add minimal code of conduct #9676 2020-03-16 12:44:28 +03:00
CONTRIBUTING.md Update CONTRIBUTING.md 2020-01-27 21:03:30 +03:00
docker-compose.yml Updated docker-compose.yml #1025 2017-07-26 20:05:32 +03:00
format_sources allow several <graphite> targets (#603) 2017-03-21 23:08:09 +04:00
LICENSE Update LICENSE 2021-01-25 17:00:13 +03:00
PreLoad.cmake Check if XCODE_IDE is true and avoid enforcing ninja in that case 2021-01-06 03:06:03 +04:00
README.md Update README.md 2021-02-13 01:46:13 +03:00
release Convert to python3 (#15007) 2020-10-02 19:54:07 +03:00
SECURITY.md Update SECURITY.md (outdated long time ago) 2021-01-21 16:48:28 +03:00
uncrustify.cfg Better .clang-format and uncrustify.cfg 2018-11-29 15:45:34 +03:00
ya.make Changes required for auto-sync with Arcadia 2020-04-16 15:31:57 +03:00

ClickHouse — open source distributed column-oriented DBMS

ClickHouse® is an open-source column-oriented database management system that allows generating analytical data reports in real time.

  • Official website has quick high-level overview of ClickHouse on main page.
  • Tutorial shows how to set up and query small ClickHouse cluster.
  • Documentation provides more in-depth information.
  • YouTube channel has a lot of content about ClickHouse in video format.
  • Slack and Telegram allow to chat with ClickHouse users in real-time.
  • Blog contains various ClickHouse-related articles, as well as announcements and reports about events.
  • Code Browser with syntax highlight and navigation.
  • Contacts can help to get your questions answered if there are any.
  • You can also fill this form to meet Yandex ClickHouse team in person.