Commit Graph

91244 Commits

Author SHA1 Message Date
Dmitry Novik
099055c183 Explicitly forbid allocations in OvercommitTracker 2022-06-21 12:26:13 +00:00
Dmitry Novik
e10f079bd3 Get rid of allocations in OvercommitTracker 2022-06-21 10:15:33 +00:00
Dmitry Novik
9ca368ac20 Use do while(false) in macro 2022-06-20 23:10:40 +00:00
Dmitry Novik
53d656d89f Fix deadlock in OvercommitTracker logging 2022-06-20 17:35:24 +00:00
Alexander Tokmakov
b5a977ad54
Merge pull request #38213 from azat/stress-thread-fuzzer-v2
tests/stress: fix TSan detection
2022-06-20 16:46:31 +03:00
Dmitry Novik
77177917bd
Merge pull request #37848 from ClickHouse/window-function-expression
Support expressions with window functions
2022-06-20 14:21:13 +02:00
Vitaly Baranov
06c4082a7c
Merge pull request #37358 from vitlibar/backup-improvements-6
Backup Improvements 6
2022-06-20 14:13:30 +02:00
Ilya Yatsishin
24d5be1669
Merge pull request #38212 from azat/no-stress
tests: disable 01646_system_restart_replicas_smoke under stress tests
2022-06-20 12:54:32 +02:00
Kseniia Sumarokova
a756b4be27
Merge pull request #37391 from azat/insert-profile-events-fix
Send profile events for INSERT queries (previously only SELECT was supported)
2022-06-20 12:16:29 +02:00
Sergei Trifonov
c0a275984f
Merge pull request #38169 from ClickHouse/trace-visualizer
improve trace-visualizer UX
2022-06-20 12:05:38 +02:00
Mikhail f. Shiryaev
e1547058cf
Merge pull request #38086 from ClickHouse/build-artifacts-upload
Build artifacts upload
2022-06-20 11:49:04 +02:00
Kseniia Sumarokova
c83594284d
Merge pull request #38205 from azat/fix-window-view
(Window View is an experimental feature) Fix LOGICAL_ERROR for WINDOW VIEW with incorrect structure
2022-06-20 11:47:45 +02:00
Nikolai Kochetov
fab62513aa
Merge pull request #38029 from ClickHouse/fix-possible-crash-after-removing-replica-in-distributed
Fix possible crash in Distributed async insert in case of removing a replica from config.
2022-06-20 11:38:35 +02:00
Vitaly Baranov
a6fc0dea4e Fix clang-tidy more. 2022-06-20 11:04:37 +02:00
Azat Khuzhin
dce71bbe06 tests/stress: fix TSan detection
+ test -f package_folder/clickhouse-client_22.7.1.1+tsan_all.deb package_folder/clickhouse-common-static-dbg_22.7.1.1+tsan_amd64.deb package_folder/clickhouse-common-static_22.7.1.1+tsan_amd64.deb package_folder/clickhouse-keeper_22.7.1.1+tsan_amd64.deb package_folder/clickhouse-server_22.7.1.1+tsan_all.deb
    /run.sh: line 13: test: too many arguments

Follow-up for: #38207 (cc @tavplubix)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-20 08:10:03 +03:00
Azat Khuzhin
3678870abf tests: disable 01646_system_restart_replicas_smoke under stress tests
This test executes SYSTEM RESTART REPLICAS, that may leave some tables
in some tests, and the problem test is
01414_mutations_and_errors_zookeeper, that has invalid values in the
table that produces the following error:

    2022.06.19 19:02:07.165320 [ 1242562 ] {} <Error> MutateFromLogEntryTask: virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 6. DB::Exception: Cannot parse string 'Hello' as UInt64: syntax error at begin of string. Note: there are toUInt64OrZero and toUInt64OrNull functions, which returns zero/NULL instead of throwing exception.: while executing 'FUNCTION _CAST(value :: 2, 'UInt64' :: 3) -> _CAST(value, 'UInt64') UInt64 : 4': (while reading from part /var/lib/clickhouse/store/700/70043200-eae1-44da-8554-0d43b7e936d7/20191002_1_1_0/): While executing MergeTreeInOrder. (CANNOT_PARSE_TEXT), Stack trace (when copying this message, always include the lines below):

Here is part of the server log that relevant for the test:

    ...
    2022.06.19 18:33:22.495867 [ 391343 ] {e0332447-5473-4653-ba8b-b976acb304a1} <Trace> InterpreterSystemQuery: Restarting replica on test_9.replicated_mutation_table
    2022.06.19 18:33:22.503462 [ 390869 ] {} <Information> test_9.replicated_mutation_table (70043200-eae1-44da-8554-0d43b7e936d7): Stopped being leader
    ...
    2022.06.19 18:33:23.396760 [ 395825 ] {09ee374d-a8d9-47db-bdca-611d605b40c6} <Error> executeQuery: Code: 341. DB::Exception: Mutation is not finished because table shutdown was called. It will be done after table restart. (UNFINISHED) (version 22.6.1.1985 (official build)) (from [::1]:40558) (comment: '01414_mutations_and_errors_zookeeper.sh') (in query: ALTER TABLE replicated_mutation_table MODIFY COLUMN value UInt64 SETTINGS replication_alter_partitions_sync = 2), Stack trace (when copying this message, always include the lines below):
    ...
    2022.06.19 18:33:23.467115 [ 390869 ] {} <Debug> test_9.replicated_mutation_table (70043200-eae1-44da-8554-0d43b7e936d7): Loading data parts
    2022.06.19 18:33:23.471062 [ 390869 ] {} <Debug> test_9.replicated_mutation_table (70043200-eae1-44da-8554-0d43b7e936d7): Loaded data parts (3 items)
    ...
    2022.06.19 18:33:23.515997 [ 390869 ] {} <Trace> test_9.replicated_mutation_table (ReplicatedMergeTreeRestartingThread): Restarting thread finished
    ...
    2022.06.19 18:33:23.522475 [ 390869 ] {} <Trace> test_9.replicated_mutation_table (PartMovesBetweenShardsOrchestrator): PartMovesBetweenShardsOrchestrator thread finished
    ...
    2022.06.19 18:33:24.960630 [ 391343 ] {e0332447-5473-4653-ba8b-b976acb304a1} <Error> executeQuery: Code: 57. DB::Exception: Cannot attach table with UUID 9b62c1d4-cf4a-4e41-bd11-bafb1446495c, because it was detached but still used by some query. Retry later. (TABLE_ALREADY_EXISTS) (version 22.6.1.1985 (official build)) (from [::1]:47448) (comment: 01646_system_restart_replicas_smoke.sql) (in query: SYSTEM RESTART REPLICAS;), Stack trace (when copying this message, always include the lines below):
    ...
    2022.06.19 18:33:24.490940 [ 400623 ] {00c29852-e786-4e53-a44a-5f1c5f23c698} <Debug> executeQuery: (from [::1]:48804) (comment: '01414_mutations_and_errors_zookeeper.sh') SELECT distinct(value) FROM replicated_mutation_table ORDER BY value (stage: Complete)
    2022.06.19 18:33:24.502168 [ 400623 ] {00c29852-e786-4e53-a44a-5f1c5f23c698} <Error> executeQuery: Code: 60. DB::Exception: Table test_9.replicated_mutation_table doesn't exist. (UNKNOWN_TABLE) (version 22.6.1.1985 (official build)) (from [::1]:48804) (comment: '01414_mutations_and_errors_zookeeper.sh') (in query: SELECT distinct(value) FROM replicated_mutation_table ORDER BY value), Stack trace (when copying this message, always include the lines below):
    ...
    2022.06.19 18:33:25.048152 [ 395940 ] {bb31a17f-aca1-411a-ab30-c6b7598c59e5} <Debug> executeQuery: (from [::1]:49236) (comment: '01414_mutations_and_errors_zookeeper.sh') DROP TABLE IF EXISTS replicated_mutation_table (stage: Complete)

And if this table will be left, then checking error messages in
/var/log/clickhouse-server/clickhouse-server.backward.clean.log will
fail, like in [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/38205/90b57e7445d5167ea2170bfe03af29faffc195cf/stress_test__undefined__actions_.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-20 08:03:22 +03:00
Azat Khuzhin
bb20cfb303 tests: drop IF EXISTS for DROP in 01414_mutations_and_errors_zookeeper
This will make troubleshooting easier.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-20 07:58:01 +03:00
Azat Khuzhin
e307a728ca tests: add no-stress tag
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-20 07:48:10 +03:00
Azat Khuzhin
e79c24deda docker/test/stress/stress: apply black
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-20 07:48:10 +03:00
Azat Khuzhin
8937785551 tests/clickhouse-test: apply black
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-20 07:43:59 +03:00
Vitaly Baranov
638ea23399 Fix build. 2022-06-20 03:44:59 +02:00
Vitaly Baranov
2c8788266c Fix tests. 2022-06-20 03:44:49 +02:00
Alexander Tokmakov
1a67740cd3
Merge pull request #38207 from azat/stress-thread-fuzzer
tests/stress: fix TSan detection (enables thread fuzzer for non-TSan builds)
2022-06-20 01:38:12 +03:00
mergify[bot]
670a63865e
Merge branch 'master' into window-function-expression 2022-06-19 22:14:54 +00:00
Alexey Milovidov
7700c26076
Merge pull request #38196 from ClickHouse/revert-38194-revert-37015-zstd_window_log_max
Revert "Revert "Add a setting to use more memory for zstd decompression""
2022-06-19 19:58:03 +03:00
Alexey Milovidov
c941d0c8e2
Merge pull request #33653 from ClickHouse/enable-some-settings
Enable some settings
2022-06-19 19:52:56 +03:00
Alexey Milovidov
a60a55b83c
Merge pull request #38203 from pan3793/patch-1
Docs: Mention Spark ClickHouse Connector in thrid party
2022-06-19 19:51:58 +03:00
Alexey Milovidov
354a10d1e1
Merge pull request #35075 from azat/dist-insert-hung-fix
Disable send_logs_level for INSERT into Distributed to avoid possible hung
2022-06-19 19:39:56 +03:00
Azat Khuzhin
d154288fc6 tests/stress: fix TSan detection (enables thread fuzzer for non-TSan builds)
Follow-up for: #38106 (cc @tavplubix @alesapin)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-19 19:27:25 +03:00
Azat Khuzhin
bc3e73d776 Fix LOGICAL_ERROR for WINDOW VIEW with incorrect structure
Caching header of the source table in the WINDOW VIEW should not be
done, since there is no ability to get notification when it had been
changed (ALTER or CREATE/DROP).

And this fires on [CI], when the following tests had been executed in
order in stress tests:
- 01050_window_view_parser_tumble (leaves wm for mt)
- 01748_partition_id_pruning (cache input_header)
- 01188_attach_table_from_path (insert into mt with wm attached and
  incorrect structure)

  [CI]: https://s3.amazonaws.com/clickhouse-test-reports/38056/109980eb275c064d08bc031bfdc14d95b9a7272b/stress_test__undefined__actions_.html

Follow-up for: #37965 (@Vxider)
Fixes: #37815
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-19 19:16:39 +03:00
Vitaly Baranov
8a7c970ce0 Fix style. 2022-06-19 15:58:26 +02:00
Vitaly Baranov
b0d19328b8 Add test for ReplicatedAccessStorage. 2022-06-19 14:14:41 +02:00
alesapin
b97b1422c9
Merge pull request #38195 from azat/tests-fix-bc-check
tests: add no-backward-compatibility-check for 02067_lost_part_s3
2022-06-19 13:42:59 +02:00
Vitaly Baranov
115be82440 DiskAccessStorage is now allowed to backup by default. 2022-06-19 13:16:36 +02:00
Vitaly Baranov
9f197defda Add support for setting 'allow_backup' to skip access entities from putting to backup. 2022-06-19 12:49:50 +02:00
Vitaly Baranov
01aaaf7395 More accurate access checking for RESTORE. 2022-06-19 11:26:41 +02:00
Alexey Milovidov
2ad6409d79
Merge pull request #38198 from ClickHouse/alexey-milovidov-patch-4
Add `userver` to the docs
2022-06-19 07:34:19 +03:00
mergify[bot]
809cf7e6e1
Merge branch 'master' into enable-some-settings 2022-06-19 04:00:11 +00:00
Alexey Milovidov
457f49a43e Add a word into spelling dictionary 2022-06-19 05:58:45 +02:00
Alexey Milovidov
04b7a34870 Maybe fix error 2022-06-19 05:21:59 +02:00
Cheng Pan
ee47ab1aa4
Docs: Mention Spark ClickHouse Connector in thrid party
It builds on Spark DataSourceV2 API and gRPC protocol.
2022-06-19 09:11:44 +08:00
Alexey Milovidov
ac0891a48d
Merge pull request #38166 from ClickHouse/grouping-docs
Update GROUP BY clause docs
2022-06-18 22:12:55 +03:00
Alexey Milovidov
67a66721f2 Merge branch 'master' into enable-some-settings 2022-06-18 19:16:23 +02:00
Alexey Milovidov
88afbe5742 Fix bad test 2022-06-18 19:12:19 +02:00
Alexey Milovidov
c8304d232e
Add userver to the docs 2022-06-18 20:08:32 +03:00
Alexey Milovidov
61aa0a6234 Merge branch 'master' into revert-38194-revert-37015-zstd_window_log_max 2022-06-18 18:27:25 +02:00
Alexey Milovidov
8c8cd6a21d
Merge pull request #38192 from den-crane/patch-16
Doc. Examples for window functions
2022-06-18 15:59:44 +03:00
Alexey Milovidov
aebc090701
Update index.md 2022-06-18 15:56:37 +03:00
Alexey Milovidov
73709b0488
Revert "Revert "Add a setting to use more memory for zstd decompression"" 2022-06-18 15:55:35 +03:00
Antonio Andelic
dd75e9cba5
Merge pull request #38149 from vdimir/multiple_joins_original_names_34697
Add setting multiple_joins_try_to_keep_original_names
2022-06-18 14:47:31 +02:00