Commit Graph

11218 Commits

Author SHA1 Message Date
Vitaly Baranov
06c4082a7c
Merge pull request #37358 from vitlibar/backup-improvements-6
Backup Improvements 6
2022-06-20 14:13:30 +02:00
Ilya Yatsishin
24d5be1669
Merge pull request #38212 from azat/no-stress
tests: disable 01646_system_restart_replicas_smoke under stress tests
2022-06-20 12:54:32 +02:00
Kseniia Sumarokova
a756b4be27
Merge pull request #37391 from azat/insert-profile-events-fix
Send profile events for INSERT queries (previously only SELECT was supported)
2022-06-20 12:16:29 +02:00
Kseniia Sumarokova
c83594284d
Merge pull request #38205 from azat/fix-window-view
(Window View is an experimental feature) Fix LOGICAL_ERROR for WINDOW VIEW with incorrect structure
2022-06-20 11:47:45 +02:00
Azat Khuzhin
3678870abf tests: disable 01646_system_restart_replicas_smoke under stress tests
This test executes SYSTEM RESTART REPLICAS, that may leave some tables
in some tests, and the problem test is
01414_mutations_and_errors_zookeeper, that has invalid values in the
table that produces the following error:

    2022.06.19 19:02:07.165320 [ 1242562 ] {} <Error> MutateFromLogEntryTask: virtual bool DB::ReplicatedMergeMutateTaskBase::executeStep(): Code: 6. DB::Exception: Cannot parse string 'Hello' as UInt64: syntax error at begin of string. Note: there are toUInt64OrZero and toUInt64OrNull functions, which returns zero/NULL instead of throwing exception.: while executing 'FUNCTION _CAST(value :: 2, 'UInt64' :: 3) -> _CAST(value, 'UInt64') UInt64 : 4': (while reading from part /var/lib/clickhouse/store/700/70043200-eae1-44da-8554-0d43b7e936d7/20191002_1_1_0/): While executing MergeTreeInOrder. (CANNOT_PARSE_TEXT), Stack trace (when copying this message, always include the lines below):

Here is part of the server log that relevant for the test:

    ...
    2022.06.19 18:33:22.495867 [ 391343 ] {e0332447-5473-4653-ba8b-b976acb304a1} <Trace> InterpreterSystemQuery: Restarting replica on test_9.replicated_mutation_table
    2022.06.19 18:33:22.503462 [ 390869 ] {} <Information> test_9.replicated_mutation_table (70043200-eae1-44da-8554-0d43b7e936d7): Stopped being leader
    ...
    2022.06.19 18:33:23.396760 [ 395825 ] {09ee374d-a8d9-47db-bdca-611d605b40c6} <Error> executeQuery: Code: 341. DB::Exception: Mutation is not finished because table shutdown was called. It will be done after table restart. (UNFINISHED) (version 22.6.1.1985 (official build)) (from [::1]:40558) (comment: '01414_mutations_and_errors_zookeeper.sh') (in query: ALTER TABLE replicated_mutation_table MODIFY COLUMN value UInt64 SETTINGS replication_alter_partitions_sync = 2), Stack trace (when copying this message, always include the lines below):
    ...
    2022.06.19 18:33:23.467115 [ 390869 ] {} <Debug> test_9.replicated_mutation_table (70043200-eae1-44da-8554-0d43b7e936d7): Loading data parts
    2022.06.19 18:33:23.471062 [ 390869 ] {} <Debug> test_9.replicated_mutation_table (70043200-eae1-44da-8554-0d43b7e936d7): Loaded data parts (3 items)
    ...
    2022.06.19 18:33:23.515997 [ 390869 ] {} <Trace> test_9.replicated_mutation_table (ReplicatedMergeTreeRestartingThread): Restarting thread finished
    ...
    2022.06.19 18:33:23.522475 [ 390869 ] {} <Trace> test_9.replicated_mutation_table (PartMovesBetweenShardsOrchestrator): PartMovesBetweenShardsOrchestrator thread finished
    ...
    2022.06.19 18:33:24.960630 [ 391343 ] {e0332447-5473-4653-ba8b-b976acb304a1} <Error> executeQuery: Code: 57. DB::Exception: Cannot attach table with UUID 9b62c1d4-cf4a-4e41-bd11-bafb1446495c, because it was detached but still used by some query. Retry later. (TABLE_ALREADY_EXISTS) (version 22.6.1.1985 (official build)) (from [::1]:47448) (comment: 01646_system_restart_replicas_smoke.sql) (in query: SYSTEM RESTART REPLICAS;), Stack trace (when copying this message, always include the lines below):
    ...
    2022.06.19 18:33:24.490940 [ 400623 ] {00c29852-e786-4e53-a44a-5f1c5f23c698} <Debug> executeQuery: (from [::1]:48804) (comment: '01414_mutations_and_errors_zookeeper.sh') SELECT distinct(value) FROM replicated_mutation_table ORDER BY value (stage: Complete)
    2022.06.19 18:33:24.502168 [ 400623 ] {00c29852-e786-4e53-a44a-5f1c5f23c698} <Error> executeQuery: Code: 60. DB::Exception: Table test_9.replicated_mutation_table doesn't exist. (UNKNOWN_TABLE) (version 22.6.1.1985 (official build)) (from [::1]:48804) (comment: '01414_mutations_and_errors_zookeeper.sh') (in query: SELECT distinct(value) FROM replicated_mutation_table ORDER BY value), Stack trace (when copying this message, always include the lines below):
    ...
    2022.06.19 18:33:25.048152 [ 395940 ] {bb31a17f-aca1-411a-ab30-c6b7598c59e5} <Debug> executeQuery: (from [::1]:49236) (comment: '01414_mutations_and_errors_zookeeper.sh') DROP TABLE IF EXISTS replicated_mutation_table (stage: Complete)

And if this table will be left, then checking error messages in
/var/log/clickhouse-server/clickhouse-server.backward.clean.log will
fail, like in [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/38205/90b57e7445d5167ea2170bfe03af29faffc195cf/stress_test__undefined__actions_.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-20 08:03:22 +03:00
Azat Khuzhin
bb20cfb303 tests: drop IF EXISTS for DROP in 01414_mutations_and_errors_zookeeper
This will make troubleshooting easier.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-20 07:58:01 +03:00
Vitaly Baranov
2c8788266c Fix tests. 2022-06-20 03:44:49 +02:00
Alexey Milovidov
7700c26076
Merge pull request #38196 from ClickHouse/revert-38194-revert-37015-zstd_window_log_max
Revert "Revert "Add a setting to use more memory for zstd decompression""
2022-06-19 19:58:03 +03:00
Alexey Milovidov
c941d0c8e2
Merge pull request #33653 from ClickHouse/enable-some-settings
Enable some settings
2022-06-19 19:52:56 +03:00
Alexey Milovidov
354a10d1e1
Merge pull request #35075 from azat/dist-insert-hung-fix
Disable send_logs_level for INSERT into Distributed to avoid possible hung
2022-06-19 19:39:56 +03:00
Azat Khuzhin
bc3e73d776 Fix LOGICAL_ERROR for WINDOW VIEW with incorrect structure
Caching header of the source table in the WINDOW VIEW should not be
done, since there is no ability to get notification when it had been
changed (ALTER or CREATE/DROP).

And this fires on [CI], when the following tests had been executed in
order in stress tests:
- 01050_window_view_parser_tumble (leaves wm for mt)
- 01748_partition_id_pruning (cache input_header)
- 01188_attach_table_from_path (insert into mt with wm attached and
  incorrect structure)

  [CI]: https://s3.amazonaws.com/clickhouse-test-reports/38056/109980eb275c064d08bc031bfdc14d95b9a7272b/stress_test__undefined__actions_.html

Follow-up for: #37965 (@Vxider)
Fixes: #37815
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-19 19:16:39 +03:00
alesapin
b97b1422c9
Merge pull request #38195 from azat/tests-fix-bc-check
tests: add no-backward-compatibility-check for 02067_lost_part_s3
2022-06-19 13:42:59 +02:00
Alexey Milovidov
67a66721f2 Merge branch 'master' into enable-some-settings 2022-06-18 19:16:23 +02:00
Alexey Milovidov
88afbe5742 Fix bad test 2022-06-18 19:12:19 +02:00
Alexey Milovidov
73709b0488
Revert "Revert "Add a setting to use more memory for zstd decompression"" 2022-06-18 15:55:35 +03:00
Antonio Andelic
dd75e9cba5
Merge pull request #38149 from vdimir/multiple_joins_original_names_34697
Add setting multiple_joins_try_to_keep_original_names
2022-06-18 14:47:31 +02:00
alesapin
0ae0cc54aa
Merge pull request #38106 from ClickHouse/tsan_less_flaky_tests
Some fixes for tests with tsan
2022-06-18 14:22:43 +02:00
Azat Khuzhin
cb552bc582 tests: add no-backward-compatibility-check for 02067_lost_part_s3
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-18 15:12:07 +03:00
alesapin
16e8b85fbf
Revert "Add a setting to use more memory for zstd decompression" 2022-06-18 14:08:14 +02:00
Alexey Milovidov
344e407f2e
Merge pull request #36914 from azat/tests-randomize
Randomize settings related to in-order read/aggregation
2022-06-18 04:21:30 +03:00
Alexey Milovidov
e20259e9ca
Merge pull request #37015 from wuxiaobai24/zstd_window_log_max
Add a setting to use more memory for zstd decompression
2022-06-18 04:19:27 +03:00
Alexey Milovidov
c107a58c11
Update 02293_test_zstd_window_log_max.sh 2022-06-18 04:18:08 +03:00
Alexander Tokmakov
828946ccf5
Update 02320_mapped_array_witn_const_nullable.sql 2022-06-17 16:44:52 +03:00
Maksim Kita
77c519ae08
Merge pull request #38142 from kitaisreal/executable-user-defined-functions-parameters-direct-fix
Disable parameters for non direct executable user defined functions
2022-06-17 12:57:27 +02:00
Alexander Tokmakov
299f102ccf
Update 02316_cast_to_ip_address_default_column.sql 2022-06-17 12:36:05 +03:00
Igor Nikonov
baebbc084f
Merge pull request #38027 from ClickHouse/decimal_rounding_fix
Fix: rounding for Decimal128/Decimal256 with more than 19-digits long scale
2022-06-17 09:48:18 +02:00
Alexander Gololobov
59d32f8c96
Merge pull request #38132 from Avogar/fix-mapped-array
Fix logical error in array mapped functions with const nullable column
2022-06-17 08:49:41 +02:00
vdimir
15988a220b
Add test multiple_joins_original_names 2022-06-16 16:00:33 +00:00
Maksim Kita
aff66742e7 Disable parameters for non direct executable user defined functions 2022-06-16 16:29:04 +02:00
Anton Popov
13ec7e3092
Merge pull request #37978 from CurtizJ/fix-sparse-s3
Fix reading of sparse columns from s3
2022-06-16 15:32:54 +02:00
mergify[bot]
2557e8ad51
Merge branch 'master' into decimal_rounding_fix 2022-06-16 10:53:49 +00:00
avogar
a3a7cc7a5d Fix logical error in array mapped functions with const nullable column 2022-06-16 10:41:53 +00:00
Alexander Tokmakov
ca649da97f better comments 2022-06-16 12:12:44 +02:00
Azat Khuzhin
b3bf7589ef Fix possible concurrent access in ProgressIndication
In case of all of the above:
- clickhouse-local
- input_format_parallel_parsing=true
- write_progress_on_update=true

It is possible concurrent access to the following:
- writeProgress() (class properties) (guarded with progress_mutex)
- thread_data/host_cpu_usage (guarded with profile_events_mutex)

v2: decrease number of rows for INSERT ProfileEvents test (10 times)
    CI: https://s3.amazonaws.com/clickhouse-test-reports/37391/4bd5c335182279dcc5020aa081b13c3044135951/stateless_tests__debug__actions__[1/3].html
v3: decrease number of rows for INSERT ProfileEvents test (10 times) and add a comment
    CI: https://s3.amazonaws.com/clickhouse-test-reports/37391/026d7f732cb166c90d6c287b02824b6c7fdebf0c/stateless_tests_flaky_check__address__actions_/runlog.log
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

f
2022-06-16 11:59:01 +03:00
Azat Khuzhin
4baa7690ae Send profile events for INSERT queries (previously only SELECT was supported)
Reproducer:

    echo "1" | clickhouse-client --query="insert into function null('foo String') format TSV" --print-profile-events --profile-events-delay-ms=-1

However, clickhouse-local is differnt, it does sent the periodically,
but only if query was long enough, i.e.:

    # yes | head -n100000 | clickhouse-local --query="insert into function null('foo String') format TSV" --print-profile-events --profile-events-delay-ms=-1
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] ContextLock: 10 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] DiskReadElapsedMicroseconds: 29 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] IOBufferAllocBytes: 200000 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] IOBufferAllocs: 1 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] InsertQuery: 1 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] InsertedBytes: 1000000 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] InsertedRows: 100000 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] MemoryTrackerUsage: 1521975 (gauge)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] OSCPUVirtualTimeMicroseconds: 102148 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] OSReadChars: 135700 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] OSWriteChars: 8 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] Query: 1 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] RWLockAcquiredReadLocks: 2 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] ReadBufferFromFileDescriptorRead: 5 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] ReadBufferFromFileDescriptorReadBytes: 134464 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] RealTimeMicroseconds: 293747 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] SoftPageFaults: 382 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] TableFunctionExecute: 1 (increment)
    [s1.ch] 2022.05.20 15:20:27 [ 0 ] UserTimeMicroseconds: 102148 (increment)

v2: Proper support ProfileEvents in INSERTs (with protocol change)
v3: Receive profile events on INSERT queries
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-16 11:59:01 +03:00
mergify[bot]
0167b90cf9
Merge branch 'master' into tests-randomize 2022-06-16 06:04:58 +00:00
mergify[bot]
ff9f4af28c
Merge branch 'master' into enable-some-settings 2022-06-16 03:51:59 +00:00
Daniel Kutenin
a769dea8ef
Merge branch 'ClickHouse:master' into master 2022-06-15 19:12:10 +01:00
Alexander Tokmakov
9cac78b498 make tests with tsan less flaky 2022-06-15 19:54:46 +02:00
Alexey Milovidov
7baa54f92f
Merge pull request #37697 from zvonand/dt64-neg-subseconds
Fix DateTime64 negative fractional seconds
2022-06-15 19:30:42 +03:00
Danila Kutenin
048f56bf4d Fix some tests and comments 2022-06-15 14:40:21 +00:00
Alexander Tokmakov
5bfb15262c Revert "More parallel execution for queries with FINAL (#36396)"
This reverts commit c8afeafe0e.
2022-06-15 17:25:38 +03:00
Yakov Olkhovskiy
e604d31feb
Merge pull request #38074 from ClickHouse/bug-with-fill-extra
Inconsistency in ORDER BY ... WITH FILL feature.
2022-06-15 08:19:55 -04:00
mergify[bot]
d704264fae
Merge branch 'master' into decimal_rounding_fix 2022-06-15 10:47:09 +00:00
Nikita Taranov
c8afeafe0e
More parallel execution for queries with FINAL (#36396) 2022-06-15 12:44:20 +02:00
Robert Schulze
9794098ebb
Merge pull request #37553 from koloshmet/fpc_codec
FPC Codec for floating point data
2022-06-15 12:03:41 +02:00
Alexander Tokmakov
dcfe4eea3c
Update 01417_freeze_partition_verbose.sh 2022-06-15 12:53:53 +03:00
Yakov Olkhovskiy
411695bd97 do not fill 'to' boundary 2022-06-14 22:26:50 -04:00
mergify[bot]
12bd2c0e3f
Merge branch 'master' into dt64-neg-subseconds 2022-06-15 02:24:17 +00:00
Alexey Milovidov
ab9fc572d5
Merge pull request #37667 from ClickHouse/group-by-enum-fix
Support types with non-standard defaults in ROLLUP, CUBE, GROUPING SETS
2022-06-15 05:14:33 +03:00