Vadim Volodin
637d293fbd
Add SYSTEM UNFREEZE query
2022-06-08 15:21:14 +03:00
alesapin
a16cf7e77e
Merge pull request #37892 from azat/mv-tests-overlaps
...
Fix stress hung because of attached MV with sleep()
2022-06-08 12:42:13 +02:00
Maksim Kita
4e160105b9
Merge pull request #37805 from kitaisreal/dictionaries-hierarchy-nullable-key-support
...
Hierarchical dictinaries support nullable parent key
2022-06-08 12:36:09 +02:00
alesapin
9fdc783eac
Merge pull request #37906 from ClickHouse/remove_tmp_directories_on_replicated
...
Remove temporary `delete_tmp_` on clickhouse server start
2022-06-08 12:35:51 +02:00
Kseniia Sumarokova
6211a1c390
Merge pull request #37801 from azat/s3-fix-empty-read
...
Fix reading of empty S3 files
2022-06-07 23:36:14 +02:00
alesapin
8b26953e3f
Merge pull request #37908 from ClickHouse/tavplubix-patch-1
...
Try fix `test_grpc_protocol/test.py::test_progress`
2022-06-07 21:48:00 +02:00
alesapin
580b02631c
Merge pull request #37890 from ClickHouse/fix_keeper_converter_tests
...
Fix keeper converter test
2022-06-07 21:45:11 +02:00
Azat Khuzhin
5d0a185cb3
Fix assertion for getImplementationBufferOffset() for Log family on S3
...
Test test_log_family_s3/test.py::test_log_family_s3[TinyLog]:
clickhouse: ./src/Disks/IO/AsynchronousReadIndirectBufferFromRemoteFS.cpp:213: virtual bool DB::AsynchronousReadIndirectBufferFromRemoteFS::nextImpl(): Assertion `file_offset_of_buffer_end == impl->getImplementationBufferOffset()' failed.
v2: fix assertion instead of adjusting file_offset_of_buffer_end in ReadBufferFromRemoteFSGather.cpp
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-07 20:41:26 +03:00
Azat Khuzhin
71285edfbd
Fix reading of empty S3 files
...
It is possible for ReadBufferFromS3::nextImpl() called even after eof(),
at least once, and in this case, if the file was empty, then local
working_buffer will be null, while impl.working_buffer will be empty,
but not null, and so local position() after impl->position() =
position() will be incorrect.
I found this with test_storage_s3/test.py::test_empty_file in debug
build, assertion catched this, so maybe it worth get back debug
integration build...
v2: fix test_log_family_s3 failures
https://s3.amazonaws.com/clickhouse-test-reports/37801/b5e6e2ddae94d6a7eac551309cb67003dff97df1/integration_tests__asan__actions__[2/3].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-07 20:41:26 +03:00
Azat Khuzhin
8ddf277670
tests: test_log_family_s3 apply black
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-07 20:41:26 +03:00
Azat Khuzhin
ee45bb3c65
tests: use id for parametrized tests in test_log_family_s3
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-07 20:41:26 +03:00
Azat Khuzhin
499ff70295
tests: improve test_log_family_s3 in case of failures
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-07 20:41:26 +03:00
Alexander Tokmakov
51e27c22a7
Merge pull request #37902 from ClickHouse/revert-37647-Fix-all-CheckTriviallyCopyableMove-Errors
...
Revert "Fix errors of CheckTriviallyCopyableMove type"
2022-06-07 19:23:19 +03:00
alesapin
96b454e1b0
BLACK
2022-06-07 17:58:53 +02:00
alesapin
e32d36d790
Proper fix
2022-06-07 17:58:32 +02:00
Alexander Tokmakov
a1e69a287f
Update test.py
2022-06-07 18:57:26 +03:00
alesapin
5741ac2b9e
Remove redundant line
2022-06-07 17:41:14 +02:00
Dmitry Novik
a3d70f27f4
Merge pull request #36944 from excitoon-favorites/better_exp_smooth
...
Fix for exponential time decaying window functions
2022-06-07 17:20:43 +02:00
alesapin
e7b76f5dee
Remove temporary directories on replicated table start
2022-06-07 16:32:29 +02:00
alesapin
278e7722ed
Review fix
2022-06-07 16:09:21 +02:00
Nikita Mikhaylov
85a1204e95
Do not calculate integrals in statistical tests ( #36953 )
2022-06-07 15:39:39 +02:00
mergify[bot]
2d01abf871
Merge branch 'master' into revert-37647-Fix-all-CheckTriviallyCopyableMove-Errors
2022-06-07 13:32:30 +00:00
Igor Nikonov
dcad154105
Merge pull request #37866 from ClickHouse/igor_minor_cleanup
...
Minor cleanup
2022-06-07 15:24:56 +02:00
Anton Popov
3af495d732
Merge pull request #37903 from ClickHouse/nickitat-patch-2
...
Fix build
2022-06-07 15:02:32 +02:00
Nikita Taranov
d09ad5fcd5
Fix build
...
https://s3.amazonaws.com/clickhouse-test-reports/36593/a4b9614385e0ecd1654d15f507159678758ec557/fast_test__actions_/build_log.txt
2022-06-07 13:56:03 +02:00
Anton Popov
df6882d2b9
Revert "Fix errors of CheckTriviallyCopyableMove type"
2022-06-07 13:53:10 +02:00
Kruglov Pavel
91c8951a1a
Merge pull request #37698 from Avogar/update_format_settings_docs
...
Refactor docs related to format settings
2022-06-07 13:28:36 +02:00
Yakov Olkhovskiy
17fbf49d13
Merge pull request #37765 from ClickHouse/bug-0leading-int
...
Number's parser refactoring, no octal support
2022-06-07 07:24:42 -04:00
alesapin
b459e33892
Merge branch 'master' into fix_keeper_converter_tests
2022-06-07 13:08:40 +02:00
Anton Popov
ef6f5a6500
Merge pull request #37570 from azat/column-ttl-expired-fix
...
Do not write expired columns by TTL after subsequent merges
2022-06-07 13:05:03 +02:00
Kseniia Sumarokova
c4b1f537cd
Merge pull request #37879 from Vxider/wv-inner-target-drop
...
Do not drop the inner target table when alter WindowView
2022-06-07 13:00:15 +02:00
Kruglov Pavel
83418ebba2
Merge pull request #37712 from vdimir/duplicate_download_previous_release
...
Get rid of duplicate download_previous_release
2022-06-07 12:51:34 +02:00
Vladimir C
1c184a7e18
Merge pull request #37840 from azat/join-query-fatal-errors
2022-06-07 12:07:37 +02:00
Kruglov Pavel
fe2e5d722c
Merge pull request #37826 from azat/remote-query-fix
...
Fix possible "No more packets are available" for distributed queries
2022-06-07 11:57:26 +02:00
Kseniia Sumarokova
edc6b68801
Merge pull request #37853 from Vxider/fire-when-inserted
...
Fire and clean windows in WindowView only when data is inserted
2022-06-07 11:24:50 +02:00
Kseniia Sumarokova
4272ca8836
Merge pull request #36060 from Varinara/master
...
Add feature disks (ls, C, list-disks, disk, help)
2022-06-07 11:22:23 +02:00
Vitaly Baranov
d199478169
Merge pull request #37303 from ClickHouse/fix_trash
...
Try to fix some trash
2022-06-07 10:17:39 +02:00
Robert Schulze
07ba122572
Merge pull request #37642 from amosbird/less-flaky-jbod-test
...
Less flaky jbod rebalancer test
2022-06-07 09:16:09 +02:00
Azat Khuzhin
078678237e
Fix possible "No more packets are available" for distributed queries
...
CI founds the following case:
<details>
2022.05.25 22:36:06.778808 [ 3037 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Fatal> : Logical error: 'No more packets are available.'.
2022.05.25 22:42:24.960075 [ 17397 ] {} <Fatal> BaseDaemon: ########################################
2022.05.25 22:42:24.971173 [ 17397 ] {} <Fatal> BaseDaemon: (version 22.6.1.1, build id: 9A1F9489854CED36) (from thread 3037) (query_id: 77743723-1fcd-4b3d-babc-d0615e3ff40e) (query: SELECT * FROM
2022.05.25 22:42:25.046871 [ 17397 ] {} <Fatal> BaseDaemon: 5. ./build_docker/../src/Common/Exception.cpp:47: DB::abortOnFailedAssertion()
2022.05.25 22:42:25.181449 [ 17397 ] {} <Fatal> BaseDaemon: 6. ./build_docker/../src/Common/Exception.cpp:70: DB::Exception::Exception()
2022.05.25 22:42:25.367710 [ 17397 ] {} <Fatal> BaseDaemon: 7. ./build_docker/../src/Client/MultiplexedConnections.cpp:0: DB::MultiplexedConnections::receivePacketUnlocked()
2022.05.25 22:42:25.414201 [ 17397 ] {} <Fatal> BaseDaemon: 8. ./build_docker/../src/Client/MultiplexedConnections.cpp:0: DB::MultiplexedConnections::receivePacket()
2022.05.25 22:42:25.493066 [ 17397 ] {} <Fatal> BaseDaemon: 9. ./build_docker/../src/QueryPipeline/RemoteQueryExecutor.cpp:279: DB::RemoteQueryExecutor::read()
2022.05.25 22:42:25.612679 [ 17397 ] {} <Fatal> BaseDaemon: 10. ./build_docker/../src/Processors/Sources/RemoteSource.cpp:0: DB::RemoteSource::tryGenerate()
Here are additional logs for this query:
$ pigz -cd clickhouse-server.stress.log.gz | fgrep -a 77743723-1fcd-4b3d-babc-d0615e3ff40e | fgrep -e Connection -e Distributed -e Fatal
2022.05.25 22:36:04.698671 [ 6613 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Trace> Connection (127.0.0.2:9000): Connecting. Database: (not specified). User: default
2022.05.25 22:36:04.722568 [ 3419 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Trace> Connection (127.0.0.2:9000): Connecting. Database: (not specified). User: default
2022.05.25 22:36:05.014432 [ 6613 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Trace> Connection (127.0.0.2:9000): Connected to ClickHouse server version 22.6.1.
2022.05.25 22:36:05.091397 [ 6613 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Debug> Connection (127.0.0.2:9000): Sent data for 2 scalars, total 2 rows in 0.000125814 sec., 15602 rows/sec., 68.00 B (517.81 KiB/sec.), compressed 0.4594594594594595 times to 148.00 B (1.10 MiB/sec.)
2022.05.25 22:36:05.301301 [ 3419 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Trace> Connection (127.0.0.2:9000): Connected to ClickHouse server version 22.6.1.
2022.05.25 22:36:05.343140 [ 3419 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Debug> Connection (127.0.0.2:9000): Sent data for 2 scalars, total 2 rows in 0.000116304 sec., 16889 rows/sec., 68.00 B (559.80 KiB/sec.), compressed 0.4594594594594595 times to 148.00 B (1.19 MiB/sec.)
2022.05.25 22:36:06.682535 [ 6613 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Trace> StorageDistributed (remote): (127.0.0.2:9000) Cancelling query because enough data has been read
2022.05.25 22:36:06.778808 [ 3037 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Fatal> : Logical error: 'No more packets are available.'.
2022.05.25 22:36:06.789505 [ 3419 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Trace> StorageDistributed (remote): (127.0.0.2:9000) Cancelling query because enough data has been read
2022.05.25 22:42:24.971173 [ 17397 ] {} <Fatal> BaseDaemon: (version 22.6.1.1, build id: 9A1F9489854CED36) (from thread 3037) (query_id: 77743723-1fcd-4b3d-babc-d0615e3ff40e) (query: SELECT * FROM
</details>
So between cancelling different sources the LOGICAL_ERROR occured, I
believe that this is because of the race:
T1: T2:
RemoteQueryExecutor::read()
checks was_cancelled
RemoteQueryExecutor::tryCancel()
connections->cancel()
calls connections->receivePacket()
Note, for this problem async_socket_for_remote/use_hedged_requests
should be disabled, and original settings was:
- --max_parallel_replicas=3
- --use_hedged_requests=false
- --allow_experimental_parallel_reading_from_replicas=3
CI: https://s3.amazonaws.com/clickhouse-test-reports/37469/41cb029ed23e77f3a108e07e6b1b1bcb03dc7fcf/stress_test__undefined__actions_/fatal_messages.txt
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-07 08:20:32 +03:00
Azat Khuzhin
155fe2bf9b
Fix stress hung because of attached MV with sleep()
...
In stress tests non-unique databases are used, and in case of non-unique
table names, materialized views may affect other tests, especially if
SELECT from underlying table contains sleep*() like in
02187_insert_values_with_mv, CI report [1].
[1]: https://s3.amazonaws.com/clickhouse-test-reports/37469/2b513acb2b8d703e737a700a9ccd5f7f323b605f/stress_test__undefined__actions_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-06-07 07:52:06 +03:00
alesapin
2329407d6f
BLACK
2022-06-07 00:42:56 +02:00
alesapin
a4632bb6ae
Fix keeper converter test
2022-06-06 22:59:34 +02:00
mergify[bot]
7a76a7a606
Merge branch 'master' into master
2022-06-06 19:44:32 +00:00
mergify[bot]
e237e44b69
Merge branch 'master' into fire-when-inserted
2022-06-06 19:22:55 +00:00
alesapin
5faaf1b553
Merge pull request #37884 from nickitat/update_protobuf_files
...
Update protobuf files for kafka and rabbitmq [fix integration tests]
2022-06-06 20:57:21 +02:00
Anton Popov
d40b23272e
Merge pull request #37755 from CurtizJ/fix-mutations-again
...
Return back #37266
2022-06-06 20:22:03 +02:00
Dmitry Novik
fe257a14e9
Merge pull request #37838 from ClickHouse/qoega-patch-2
...
Space in {} shows it on final page
2022-06-06 19:51:20 +02:00
Maksim Kita
612172def5
Merge pull request #37720 from kitaisreal/executable-user-defined-functions-parameters
...
Executable user defined functions support parameters
2022-06-06 16:51:56 +02:00
Varinara
1f8fab41f1
Fixe2: allow config path at any position
2022-06-06 16:52:58 +03:00
Varinara
6aa9e5d649
Fixe: allow config path at any position
2022-06-06 16:52:58 +03:00