thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 02:41:59 +00:00
Code Issues Packages Projects Releases Wiki Activity
91,082 Commits 9,328 Branches 1,783 Tags 2.1 GiB
ea75ae8653
Commit Graph

91082 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
alesapin
580b02631c
Merge pull request #37890 from ClickHouse/fix_keeper_converter_tests 
Fix keeper converter test
 2022-06-07 21:45:11 +02:00
KinderRiven
ff16d5555c fix build 2022-06-08 03:23:40 +08:00
KinderRiven
1e47ae1026 fix build 2022-06-08 03:22:58 +08:00
KinderRiven
5e8a3c6352 fix build 2022-06-08 03:19:07 +08:00
KinderRiven
83452756bd add stateless test 2022-06-08 02:05:48 +08:00
KinderRiven
55c390c86f fix 2022-06-08 01:42:46 +08:00
Azat Khuzhin
5d0a185cb3 Fix assertion for getImplementationBufferOffset() for Log family on S3 
Test test_log_family_s3/test.py::test_log_family_s3[TinyLog]:

    clickhouse: ./src/Disks/IO/AsynchronousReadIndirectBufferFromRemoteFS.cpp:213: virtual bool DB::AsynchronousReadIndirectBufferFromRemoteFS::nextImpl(): Assertion `file_offset_of_buffer_end == impl->getImplementationBufferOffset()' failed.

v2: fix assertion instead of adjusting file_offset_of_buffer_end in ReadBufferFromRemoteFSGather.cpp
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-07 20:41:26 +03:00
Azat Khuzhin
71285edfbd Fix reading of empty S3 files 
It is possible for ReadBufferFromS3::nextImpl() called even after eof(),
at least once, and in this case, if the file was empty, then local
working_buffer will be null, while impl.working_buffer will be empty,
but not null, and so local position() after impl->position() =
position() will be incorrect.

I found this with test_storage_s3/test.py::test_empty_file in debug
build, assertion catched this, so maybe it worth get back debug
integration build...

v2: fix test_log_family_s3 failures
    https://s3.amazonaws.com/clickhouse-test-reports/37801/b5e6e2ddae94d6a7eac551309cb67003dff97df1/integration_tests__asan__actions__[2/3].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-07 20:41:26 +03:00
Azat Khuzhin
8ddf277670 tests: test_log_family_s3 apply black 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-07 20:41:26 +03:00
Azat Khuzhin
ee45bb3c65 tests: use id for parametrized tests in test_log_family_s3 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-07 20:41:26 +03:00
Azat Khuzhin
499ff70295 tests: improve test_log_family_s3 in case of failures 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-07 20:41:26 +03:00
Alexander Tokmakov
51e27c22a7
Merge pull request #37902 from ClickHouse/revert-37647-Fix-all-CheckTriviallyCopyableMove-Errors 
Revert "Fix errors of CheckTriviallyCopyableMove type"
 2022-06-07 19:23:19 +03:00
alesapin
96b454e1b0 BLACK 2022-06-07 17:58:53 +02:00
alesapin
e32d36d790 Proper fix 2022-06-07 17:58:32 +02:00
Alexander Tokmakov
a1e69a287f
Update test.py 2022-06-07 18:57:26 +03:00
alesapin
5741ac2b9e Remove redundant line 2022-06-07 17:41:14 +02:00
Dmitry Novik
a3d70f27f4
Merge pull request #36944 from excitoon-favorites/better_exp_smooth 
Fix for exponential time decaying window functions
 2022-06-07 17:20:43 +02:00
alesapin
e7b76f5dee Remove temporary directories on replicated table start 2022-06-07 16:32:29 +02:00
alesapin
278e7722ed Review fix 2022-06-07 16:09:21 +02:00
Nikita Mikhaylov
85a1204e95
Do not calculate integrals in statistical tests (#36953) 2022-06-07 15:39:39 +02:00
Bharat Nallan Chakravarthy
4855794246 Merge remote-tracking branch 'upstream/master' into ncb/build-options-ssl-version 2022-06-07 19:02:50 +05:30
mergify[bot]
2d01abf871
Merge branch 'master' into revert-37647-Fix-all-CheckTriviallyCopyableMove-Errors 2022-06-07 13:32:30 +00:00
Igor Nikonov
dcad154105
Merge pull request #37866 from ClickHouse/igor_minor_cleanup 
Minor cleanup
 2022-06-07 15:24:56 +02:00
Anton Popov
3af495d732
Merge pull request #37903 from ClickHouse/nickitat-patch-2 
Fix build
 2022-06-07 15:02:32 +02:00
Nikita Taranov
d09ad5fcd5
Fix build 
https://s3.amazonaws.com/clickhouse-test-reports/36593/a4b9614385e0ecd1654d15f507159678758ec557/fast_test__actions_/build_log.txt
 2022-06-07 13:56:03 +02:00
KinderRiven
6f8abd0328 fix 2022-06-07 19:53:24 +08:00
Anton Popov
df6882d2b9
Revert "Fix errors of CheckTriviallyCopyableMove type" 2022-06-07 13:53:10 +02:00
KinderRiven
de45f0a38c fix unit_test 2022-06-07 19:30:39 +08:00
Kruglov Pavel
91c8951a1a
Merge pull request #37698 from Avogar/update_format_settings_docs 
Refactor docs related to format settings
 2022-06-07 13:28:36 +02:00
Yakov Olkhovskiy
17fbf49d13
Merge pull request #37765 from ClickHouse/bug-0leading-int 
Number's parser refactoring, no octal support
 2022-06-07 07:24:42 -04:00
KinderRiven
f00914430d fix holder 2022-06-07 19:10:57 +08:00
alesapin
b459e33892 Merge branch 'master' into fix_keeper_converter_tests 2022-06-07 13:08:40 +02:00
Anton Popov
ef6f5a6500
Merge pull request #37570 from azat/column-ttl-expired-fix 
Do not write expired columns by TTL after subsequent merges
 2022-06-07 13:05:03 +02:00
Yakov Olkhovskiy
3d8f00a55f
Update utils/self-extr-exec/decompressor.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2022-06-07 07:04:57 -04:00
Yakov Olkhovskiy
16a28bf87a
Update utils/self-extr-exec/decompressor.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2022-06-07 07:04:39 -04:00
Kseniia Sumarokova
c4b1f537cd
Merge pull request #37879 from Vxider/wv-inner-target-drop 
Do not drop the inner target table when alter WindowView
 2022-06-07 13:00:15 +02:00
Kruglov Pavel
83418ebba2
Merge pull request #37712 from vdimir/duplicate_download_previous_release 
Get rid of duplicate download_previous_release
 2022-06-07 12:51:34 +02:00
Vladimir C
1c184a7e18
Merge pull request #37840 from azat/join-query-fatal-errors 2022-06-07 12:07:37 +02:00
Kruglov Pavel
fe2e5d722c
Merge pull request #37826 from azat/remote-query-fix 
Fix possible "No more packets are available" for distributed queries
 2022-06-07 11:57:26 +02:00
Kseniia Sumarokova
edc6b68801
Merge pull request #37853 from Vxider/fire-when-inserted 
Fire and clean windows in WindowView only when data is inserted
 2022-06-07 11:24:50 +02:00
Kseniia Sumarokova
4272ca8836
Merge pull request #36060 from Varinara/master 
Add feature disks (ls, C, list-disks, disk, help)
 2022-06-07 11:22:23 +02:00
Antonio Andelic
f25098e48d Merge branch 'master' into revert-37534-revert-37036-keeper-preprocess-operations 2022-06-07 09:11:41 +00:00
Vitaly Baranov
d199478169
Merge pull request #37303 from ClickHouse/fix_trash 
Try to fix some trash
 2022-06-07 10:17:39 +02:00
Antonio Andelic
d37ee2e221 Merge branch 'master' into keeper-broken-to-detached 2022-06-07 07:39:46 +00:00
Robert Schulze
07ba122572
Merge pull request #37642 from amosbird/less-flaky-jbod-test 
Less flaky jbod rebalancer test
 2022-06-07 09:16:09 +02:00
Azat Khuzhin
078678237e Fix possible "No more packets are available" for distributed queries 
CI founds the following case:

<details>

    2022.05.25 22:36:06.778808 [ 3037 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Fatal> : Logical error: 'No more packets are available.'.
    2022.05.25 22:42:24.960075 [ 17397 ] {} <Fatal> BaseDaemon: ########################################
    2022.05.25 22:42:24.971173 [ 17397 ] {} <Fatal> BaseDaemon: (version 22.6.1.1, build id: 9A1F9489854CED36) (from thread 3037) (query_id: 77743723-1fcd-4b3d-babc-d0615e3ff40e) (query: SELECT * FROM
    2022.05.25 22:42:25.046871 [ 17397 ] {} <Fatal> BaseDaemon: 5. ./build_docker/../src/Common/Exception.cpp:47: DB::abortOnFailedAssertion()
    2022.05.25 22:42:25.181449 [ 17397 ] {} <Fatal> BaseDaemon: 6. ./build_docker/../src/Common/Exception.cpp:70: DB::Exception::Exception()
    2022.05.25 22:42:25.367710 [ 17397 ] {} <Fatal> BaseDaemon: 7. ./build_docker/../src/Client/MultiplexedConnections.cpp:0: DB::MultiplexedConnections::receivePacketUnlocked()
    2022.05.25 22:42:25.414201 [ 17397 ] {} <Fatal> BaseDaemon: 8. ./build_docker/../src/Client/MultiplexedConnections.cpp:0: DB::MultiplexedConnections::receivePacket()
    2022.05.25 22:42:25.493066 [ 17397 ] {} <Fatal> BaseDaemon: 9. ./build_docker/../src/QueryPipeline/RemoteQueryExecutor.cpp:279: DB::RemoteQueryExecutor::read()
    2022.05.25 22:42:25.612679 [ 17397 ] {} <Fatal> BaseDaemon: 10. ./build_docker/../src/Processors/Sources/RemoteSource.cpp:0: DB::RemoteSource::tryGenerate()

Here are additional logs for this query:
    $ pigz -cd clickhouse-server.stress.log.gz | fgrep -a 77743723-1fcd-4b3d-babc-d0615e3ff40e | fgrep -e Connection -e Distributed -e Fatal
    2022.05.25 22:36:04.698671 [ 6613 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Trace> Connection (127.0.0.2:9000): Connecting. Database: (not specified). User: default
    2022.05.25 22:36:04.722568 [ 3419 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Trace> Connection (127.0.0.2:9000): Connecting. Database: (not specified). User: default
    2022.05.25 22:36:05.014432 [ 6613 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Trace> Connection (127.0.0.2:9000): Connected to ClickHouse server version 22.6.1.
    2022.05.25 22:36:05.091397 [ 6613 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Debug> Connection (127.0.0.2:9000): Sent data for 2 scalars, total 2 rows in 0.000125814 sec., 15602 rows/sec., 68.00 B (517.81 KiB/sec.), compressed 0.4594594594594595 times to 148.00 B (1.10 MiB/sec.)
    2022.05.25 22:36:05.301301 [ 3419 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Trace> Connection (127.0.0.2:9000): Connected to ClickHouse server version 22.6.1.
    2022.05.25 22:36:05.343140 [ 3419 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Debug> Connection (127.0.0.2:9000): Sent data for 2 scalars, total 2 rows in 0.000116304 sec., 16889 rows/sec., 68.00 B (559.80 KiB/sec.), compressed 0.4594594594594595 times to 148.00 B (1.19 MiB/sec.)
    2022.05.25 22:36:06.682535 [ 6613 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Trace> StorageDistributed (remote): (127.0.0.2:9000) Cancelling query because enough data has been read
    2022.05.25 22:36:06.778808 [ 3037 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Fatal> : Logical error: 'No more packets are available.'.
    2022.05.25 22:36:06.789505 [ 3419 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Trace> StorageDistributed (remote): (127.0.0.2:9000) Cancelling query because enough data has been read
    2022.05.25 22:42:24.971173 [ 17397 ] {} <Fatal> BaseDaemon: (version 22.6.1.1, build id: 9A1F9489854CED36) (from thread 3037) (query_id: 77743723-1fcd-4b3d-babc-d0615e3ff40e) (query: SELECT * FROM

</details>

So between cancelling different sources the LOGICAL_ERROR occured, I
believe that this is because of the race:

    T1:                                        T2:
    RemoteQueryExecutor::read()
    checks was_cancelled
                                               RemoteQueryExecutor::tryCancel()
                                               connections->cancel()
    calls connections->receivePacket()

Note, for this problem async_socket_for_remote/use_hedged_requests
should be disabled, and original settings was:
- --max_parallel_replicas=3
- --use_hedged_requests=false
- --allow_experimental_parallel_reading_from_replicas=3

CI: https://s3.amazonaws.com/clickhouse-test-reports/37469/41cb029ed23e77f3a108e07e6b1b1bcb03dc7fcf/stress_test__undefined__actions_/fatal_messages.txt
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-07 08:20:32 +03:00
Azat Khuzhin
155fe2bf9b Fix stress hung because of attached MV with sleep() 
In stress tests non-unique databases are used, and in case of non-unique
table names, materialized views may affect other tests, especially if
SELECT from underlying table contains sleep*() like in
02187_insert_values_with_mv, CI report [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/37469/2b513acb2b8d703e737a700a9ccd5f7f323b605f/stress_test__undefined__actions_.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-07 07:52:06 +03:00
taiyang-li
73a484256e Merge branch 'master' into async_hdfs_read_buffer 2022-06-07 12:16:46 +08:00
KinderRiven
3b2a427002 fix 2022-06-07 11:50:56 +08:00
taiyang-li
c65c56fd48 fix typo 2022-06-07 09:58:29 +08:00