thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-25 09:02:00 +00:00
Code Issues Packages Projects Releases Wiki Activity
90,450 Commits 9,170 Branches 1,781 Tags 2.1 GiB
35c3944d1a
Commit Graph

90450 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Robert Schulze
35c3944d1a
Warn when gold is used instead of lld 2022-06-13 11:31:08 +00:00
Robert Schulze
0696ee3758
Always disable --color-diagnostics for LLVM 
The flag is not only not recognized on MacOS, it is not recognized by
the gold linker in general.
 2022-06-10 10:39:38 +02:00
Kseniia Sumarokova
6211a1c390
Merge pull request #37801 from azat/s3-fix-empty-read 
Fix reading of empty S3 files
 2022-06-07 23:36:14 +02:00
alesapin
8b26953e3f
Merge pull request #37908 from ClickHouse/tavplubix-patch-1 
Try fix `test_grpc_protocol/test.py::test_progress`
 2022-06-07 21:48:00 +02:00
alesapin
580b02631c
Merge pull request #37890 from ClickHouse/fix_keeper_converter_tests 
Fix keeper converter test
 2022-06-07 21:45:11 +02:00
Azat Khuzhin
5d0a185cb3 Fix assertion for getImplementationBufferOffset() for Log family on S3 
Test test_log_family_s3/test.py::test_log_family_s3[TinyLog]:

    clickhouse: ./src/Disks/IO/AsynchronousReadIndirectBufferFromRemoteFS.cpp:213: virtual bool DB::AsynchronousReadIndirectBufferFromRemoteFS::nextImpl(): Assertion `file_offset_of_buffer_end == impl->getImplementationBufferOffset()' failed.

v2: fix assertion instead of adjusting file_offset_of_buffer_end in ReadBufferFromRemoteFSGather.cpp
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-07 20:41:26 +03:00
Azat Khuzhin
71285edfbd Fix reading of empty S3 files 
It is possible for ReadBufferFromS3::nextImpl() called even after eof(),
at least once, and in this case, if the file was empty, then local
working_buffer will be null, while impl.working_buffer will be empty,
but not null, and so local position() after impl->position() =
position() will be incorrect.

I found this with test_storage_s3/test.py::test_empty_file in debug
build, assertion catched this, so maybe it worth get back debug
integration build...

v2: fix test_log_family_s3 failures
    https://s3.amazonaws.com/clickhouse-test-reports/37801/b5e6e2ddae94d6a7eac551309cb67003dff97df1/integration_tests__asan__actions__[2/3].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-07 20:41:26 +03:00
Azat Khuzhin
8ddf277670 tests: test_log_family_s3 apply black 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-07 20:41:26 +03:00
Azat Khuzhin
ee45bb3c65 tests: use id for parametrized tests in test_log_family_s3 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-07 20:41:26 +03:00
Azat Khuzhin
499ff70295 tests: improve test_log_family_s3 in case of failures 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-07 20:41:26 +03:00
Alexander Tokmakov
51e27c22a7
Merge pull request #37902 from ClickHouse/revert-37647-Fix-all-CheckTriviallyCopyableMove-Errors 
Revert "Fix errors of CheckTriviallyCopyableMove type"
 2022-06-07 19:23:19 +03:00
Alexander Tokmakov
a1e69a287f
Update test.py 2022-06-07 18:57:26 +03:00
Dmitry Novik
a3d70f27f4
Merge pull request #36944 from excitoon-favorites/better_exp_smooth 
Fix for exponential time decaying window functions
 2022-06-07 17:20:43 +02:00
alesapin
278e7722ed Review fix 2022-06-07 16:09:21 +02:00
Nikita Mikhaylov
85a1204e95
Do not calculate integrals in statistical tests (#36953) 2022-06-07 15:39:39 +02:00
mergify[bot]
2d01abf871
Merge branch 'master' into revert-37647-Fix-all-CheckTriviallyCopyableMove-Errors 2022-06-07 13:32:30 +00:00
Igor Nikonov
dcad154105
Merge pull request #37866 from ClickHouse/igor_minor_cleanup 
Minor cleanup
 2022-06-07 15:24:56 +02:00
Anton Popov
3af495d732
Merge pull request #37903 from ClickHouse/nickitat-patch-2 
Fix build
 2022-06-07 15:02:32 +02:00
Nikita Taranov
d09ad5fcd5
Fix build 
https://s3.amazonaws.com/clickhouse-test-reports/36593/a4b9614385e0ecd1654d15f507159678758ec557/fast_test__actions_/build_log.txt
 2022-06-07 13:56:03 +02:00
Anton Popov
df6882d2b9
Revert "Fix errors of CheckTriviallyCopyableMove type" 2022-06-07 13:53:10 +02:00
Kruglov Pavel
91c8951a1a
Merge pull request #37698 from Avogar/update_format_settings_docs 
Refactor docs related to format settings
 2022-06-07 13:28:36 +02:00
Yakov Olkhovskiy
17fbf49d13
Merge pull request #37765 from ClickHouse/bug-0leading-int 
Number's parser refactoring, no octal support
 2022-06-07 07:24:42 -04:00
alesapin
b459e33892 Merge branch 'master' into fix_keeper_converter_tests 2022-06-07 13:08:40 +02:00
Anton Popov
ef6f5a6500
Merge pull request #37570 from azat/column-ttl-expired-fix 
Do not write expired columns by TTL after subsequent merges
 2022-06-07 13:05:03 +02:00
Kseniia Sumarokova
c4b1f537cd
Merge pull request #37879 from Vxider/wv-inner-target-drop 
Do not drop the inner target table when alter WindowView
 2022-06-07 13:00:15 +02:00
Kruglov Pavel
83418ebba2
Merge pull request #37712 from vdimir/duplicate_download_previous_release 
Get rid of duplicate download_previous_release
 2022-06-07 12:51:34 +02:00
Vladimir C
1c184a7e18
Merge pull request #37840 from azat/join-query-fatal-errors 2022-06-07 12:07:37 +02:00
Kruglov Pavel
fe2e5d722c
Merge pull request #37826 from azat/remote-query-fix 
Fix possible "No more packets are available" for distributed queries
 2022-06-07 11:57:26 +02:00
Kseniia Sumarokova
edc6b68801
Merge pull request #37853 from Vxider/fire-when-inserted 
Fire and clean windows in WindowView only when data is inserted
 2022-06-07 11:24:50 +02:00
Kseniia Sumarokova
4272ca8836
Merge pull request #36060 from Varinara/master 
Add feature disks (ls, C, list-disks, disk, help)
 2022-06-07 11:22:23 +02:00
Vitaly Baranov
d199478169
Merge pull request #37303 from ClickHouse/fix_trash 
Try to fix some trash
 2022-06-07 10:17:39 +02:00
Robert Schulze
07ba122572
Merge pull request #37642 from amosbird/less-flaky-jbod-test 
Less flaky jbod rebalancer test
 2022-06-07 09:16:09 +02:00
Azat Khuzhin
078678237e Fix possible "No more packets are available" for distributed queries 
CI founds the following case:

<details>

    2022.05.25 22:36:06.778808 [ 3037 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Fatal> : Logical error: 'No more packets are available.'.
    2022.05.25 22:42:24.960075 [ 17397 ] {} <Fatal> BaseDaemon: ########################################
    2022.05.25 22:42:24.971173 [ 17397 ] {} <Fatal> BaseDaemon: (version 22.6.1.1, build id: 9A1F9489854CED36) (from thread 3037) (query_id: 77743723-1fcd-4b3d-babc-d0615e3ff40e) (query: SELECT * FROM
    2022.05.25 22:42:25.046871 [ 17397 ] {} <Fatal> BaseDaemon: 5. ./build_docker/../src/Common/Exception.cpp:47: DB::abortOnFailedAssertion()
    2022.05.25 22:42:25.181449 [ 17397 ] {} <Fatal> BaseDaemon: 6. ./build_docker/../src/Common/Exception.cpp:70: DB::Exception::Exception()
    2022.05.25 22:42:25.367710 [ 17397 ] {} <Fatal> BaseDaemon: 7. ./build_docker/../src/Client/MultiplexedConnections.cpp:0: DB::MultiplexedConnections::receivePacketUnlocked()
    2022.05.25 22:42:25.414201 [ 17397 ] {} <Fatal> BaseDaemon: 8. ./build_docker/../src/Client/MultiplexedConnections.cpp:0: DB::MultiplexedConnections::receivePacket()
    2022.05.25 22:42:25.493066 [ 17397 ] {} <Fatal> BaseDaemon: 9. ./build_docker/../src/QueryPipeline/RemoteQueryExecutor.cpp:279: DB::RemoteQueryExecutor::read()
    2022.05.25 22:42:25.612679 [ 17397 ] {} <Fatal> BaseDaemon: 10. ./build_docker/../src/Processors/Sources/RemoteSource.cpp:0: DB::RemoteSource::tryGenerate()

Here are additional logs for this query:
    $ pigz -cd clickhouse-server.stress.log.gz | fgrep -a 77743723-1fcd-4b3d-babc-d0615e3ff40e | fgrep -e Connection -e Distributed -e Fatal
    2022.05.25 22:36:04.698671 [ 6613 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Trace> Connection (127.0.0.2:9000): Connecting. Database: (not specified). User: default
    2022.05.25 22:36:04.722568 [ 3419 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Trace> Connection (127.0.0.2:9000): Connecting. Database: (not specified). User: default
    2022.05.25 22:36:05.014432 [ 6613 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Trace> Connection (127.0.0.2:9000): Connected to ClickHouse server version 22.6.1.
    2022.05.25 22:36:05.091397 [ 6613 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Debug> Connection (127.0.0.2:9000): Sent data for 2 scalars, total 2 rows in 0.000125814 sec., 15602 rows/sec., 68.00 B (517.81 KiB/sec.), compressed 0.4594594594594595 times to 148.00 B (1.10 MiB/sec.)
    2022.05.25 22:36:05.301301 [ 3419 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Trace> Connection (127.0.0.2:9000): Connected to ClickHouse server version 22.6.1.
    2022.05.25 22:36:05.343140 [ 3419 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Debug> Connection (127.0.0.2:9000): Sent data for 2 scalars, total 2 rows in 0.000116304 sec., 16889 rows/sec., 68.00 B (559.80 KiB/sec.), compressed 0.4594594594594595 times to 148.00 B (1.19 MiB/sec.)
    2022.05.25 22:36:06.682535 [ 6613 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Trace> StorageDistributed (remote): (127.0.0.2:9000) Cancelling query because enough data has been read
    2022.05.25 22:36:06.778808 [ 3037 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Fatal> : Logical error: 'No more packets are available.'.
    2022.05.25 22:36:06.789505 [ 3419 ] {77743723-1fcd-4b3d-babc-d0615e3ff40e} <Trace> StorageDistributed (remote): (127.0.0.2:9000) Cancelling query because enough data has been read
    2022.05.25 22:42:24.971173 [ 17397 ] {} <Fatal> BaseDaemon: (version 22.6.1.1, build id: 9A1F9489854CED36) (from thread 3037) (query_id: 77743723-1fcd-4b3d-babc-d0615e3ff40e) (query: SELECT * FROM

</details>

So between cancelling different sources the LOGICAL_ERROR occured, I
believe that this is because of the race:

    T1:                                        T2:
    RemoteQueryExecutor::read()
    checks was_cancelled
                                               RemoteQueryExecutor::tryCancel()
                                               connections->cancel()
    calls connections->receivePacket()

Note, for this problem async_socket_for_remote/use_hedged_requests
should be disabled, and original settings was:
- --max_parallel_replicas=3
- --use_hedged_requests=false
- --allow_experimental_parallel_reading_from_replicas=3

CI: https://s3.amazonaws.com/clickhouse-test-reports/37469/41cb029ed23e77f3a108e07e6b1b1bcb03dc7fcf/stress_test__undefined__actions_/fatal_messages.txt
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-07 08:20:32 +03:00
alesapin
2329407d6f BLACK 2022-06-07 00:42:56 +02:00
alesapin
a4632bb6ae Fix keeper converter test 2022-06-06 22:59:34 +02:00
mergify[bot]
7a76a7a606
Merge branch 'master' into master 2022-06-06 19:44:32 +00:00
mergify[bot]
e237e44b69
Merge branch 'master' into fire-when-inserted 2022-06-06 19:22:55 +00:00
alesapin
5faaf1b553
Merge pull request #37884 from nickitat/update_protobuf_files 
Update protobuf files for kafka and rabbitmq [fix integration tests]
 2022-06-06 20:57:21 +02:00
Anton Popov
d40b23272e
Merge pull request #37755 from CurtizJ/fix-mutations-again 
Return back #37266
 2022-06-06 20:22:03 +02:00
Dmitry Novik
fe257a14e9
Merge pull request #37838 from ClickHouse/qoega-patch-2 
Space in {} shows it on final page
 2022-06-06 19:51:20 +02:00
Maksim Kita
612172def5
Merge pull request #37720 from kitaisreal/executable-user-defined-functions-parameters 
Executable user defined functions support parameters
 2022-06-06 16:51:56 +02:00
Varinara
1f8fab41f1 Fixe2: allow config path at any position 2022-06-06 16:52:58 +03:00
Varinara
6aa9e5d649 Fixe: allow config path at any position 2022-06-06 16:52:58 +03:00
Varinara
e35d1460bf Corrections 2022-06-06 16:52:58 +03:00
Varinara
a330d3cb4a Corrections 2022-06-06 16:52:58 +03:00
Varinara
f5d4e9ccc1 Corrections 2022-06-06 16:52:58 +03:00
Varinara
354ef2e3b8 Corrections 2022-06-06 16:52:58 +03:00
Varinara
51409703e0 Corrections 2022-06-06 16:52:58 +03:00
Varinara
ed6e8176fe Add basic commands for disk tool (list-disks, list, move, remove, link, copy, read, write) + tests 2022-06-06 16:52:58 +03:00
Nickita Taranov
9fd9836237 update other files 2022-06-06 15:31:20 +02:00