thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-29 11:02:08 +00:00
Code Issues Packages Projects Releases Wiki Activity
60,609 Commits 9,189 Branches 1,782 Tags 2.2 GiB
a91f505899
Commit Graph

60609 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Azat Khuzhin
9db74c4dc0 Add stress test for distributed queries 
It may founds issue like in [1]:

    2021.03.18 19:05:38.783328 [ 245 ] {4b1f5ec0-bf2d-478c-a2e1-d312531db206} <Debug> executeQuery: (from 127.0.0.1:40918, using production parser) select * from dist where key = 0;
    2021.03.18 19:05:38.783760 [ 245 ] {4b1f5ec0-bf2d-478c-a2e1-d312531db206} <Debug> StorageDistributed (dist): Skipping irrelevant shards - the query will be sent to the following shards of the cluster (shard numbers): [1]
    2021.03.18 19:05:38.784012 [ 245 ] {4b1f5ec0-bf2d-478c-a2e1-d312531db206} <Trace> ContextAccess (default): Access granted: SELECT(key) ON default.dist
    2021.03.18 19:05:38.784410 [ 245 ] {4b1f5ec0-bf2d-478c-a2e1-d312531db206} <Trace> ContextAccess (default): Access granted: SELECT(key) ON default.dist
    2021.03.18 19:05:38.784488 [ 245 ] {4b1f5ec0-bf2d-478c-a2e1-d312531db206} <Trace> StorageDistributed (dist): Disabling force_optimize_skip_unused_shards for nested queries (force_optimize_skip_unused_shards_nesting exceeded)
    2021.03.18 19:05:38.784572 [ 245 ] {4b1f5ec0-bf2d-478c-a2e1-d312531db206} <Trace> InterpreterSelectQuery: Complete -> Complete
    2021.03.18 19:05:38.819063 [ 245 ] {4b1f5ec0-bf2d-478c-a2e1-d312531db206} <Information> executeQuery: Read 20 rows, 80.00 B in 0.035687783 sec., 560 rows/sec., 2.19 KiB/sec.
    2021.03.18 19:05:38.827842 [ 245 ] {4b1f5ec0-bf2d-478c-a2e1-d312531db206} <Debug> MemoryTracker: Peak memory usage (for query): 0.00 B.

    2021.03.18 19:05:38.867752 [ 547 ] {} <Fatal> BaseDaemon: ########################################
    2021.03.18 19:05:38.867959 [ 547 ] {} <Fatal> BaseDaemon: (version 21.4.1.1, build id: A0ADEC175BD65E58EA012C47C265E661C32D23B5) (from thread 245) (query_id: 4b1f5ec0-bf2d-478c-a2e1-d312531db206) Received signal Aborted (6)
    2021.03.18 19:05:38.868733 [ 547 ] {} <Fatal> BaseDaemon:
    2021.03.18 19:05:38.868958 [ 547 ] {} <Fatal> BaseDaemon: Stack trace: 0x7fd1394be18b 0x7fd13949d859 0x10c4c99b 0xd434ee1 0xd434f1a
    2021.03.18 19:05:38.870135 [ 547 ] {} <Fatal> BaseDaemon: 3. gsignal @ 0x4618b in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    2021.03.18 19:05:38.870383 [ 547 ] {} <Fatal> BaseDaemon: 4. abort @ 0x25859 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    2021.03.18 19:05:38.886783 [ 547 ] {} <Fatal> BaseDaemon: 5. /work3/azat/ch/clickhouse/.cmake/../contrib/libunwind/src/UnwindLevel1.c:396: _Unwind_Resume @ 0x10c4c99b in /usr/bin/clickhouse
    2021.03.18 19:05:47.200208 [ 547 ] {} <Fatal> BaseDaemon: 6. ? @ 0xd434ee1 in /usr/bin/clickhouse
    2021.03.18 19:05:47.348738 [ 547 ] {} <Fatal> BaseDaemon: 7.1. inlined from /work3/azat/ch/clickhouse/.cmake/../contrib/boost/boost/context/fiber_fcontext.hpp:253: boost::context::fiber::~fiber()
    2021.03.18 19:05:47.349118 [ 547 ] {} <Fatal> BaseDaemon: 7.2. inlined from ../contrib/boost/boost/context/fiber_fcontext.hpp:252: boost::context::detail::fiber_record<boost::context::fiber, FiberStack&, DB::RemoteQueryExecutorRoutine>::run(void*)
    2021.03.18 19:05:47.349163 [ 547 ] {} <Fatal> BaseDaemon: 7. ../contrib/boost/boost/context/fiber_fcontext.hpp:80: void boost::context::detail::fiber_entry<boost::context::detail::fiber_record<boost::context::fiber, FiberStack&, DB::RemoteQueryExecutorRoutine> >(boost::context::detail::transfer_t) @ 0xd434f1a in /usr/bin/clickhouse
    2021.03.18 19:05:47.618174 [ 547 ] {} <Fatal> BaseDaemon: Calculated checksum of the binary: FF3BA83D0CD648741EEEC242CB1966D9. There is no information about the reference checksum.

  [1]: https://clickhouse-test-reports.s3.yandex.net/0/1b2ed51ff5e4a3dc45567d4967108f43f680c884/stress_test_(debug).html#fail1
 2021-03-26 01:04:48 +03:00
Alexander Kuzmenkov
f3ca9db832 forgot abs() 2021-03-26 01:00:06 +03:00
Alexander Kuzmenkov
a12cc5f559 adjust perf test thresholds 2021-03-26 00:00:18 +03:00
Azat Khuzhin
b68517f69e Fix cutToFirstSignificantSubdomainCustom()/firstSignificantSubdomainCustom() for 3+level domains 
Custom TLD lists (added in #17748), may contain domain of the 3-d level,
however builtin TLD lists does not have such records, so it is not
affected.

Note that this will significantly increase hashtable lookups.

Fixes: #17748
 2021-03-26 00:00:16 +03:00
Azat Khuzhin
30cd1c6145 Fix typo in FirstSignificantSubdomainCustomLookup name 2021-03-26 00:00:16 +03:00
Azat Khuzhin
f1907acbcd Use finalize() over next() for nested writers 
Refs: https://github.com/ClickHouse/ClickHouse/pull/21325#discussion_r585348309
 2021-03-25 23:42:58 +03:00
Azat Khuzhin
50003e496a Use existing logger for logging from WriteBufferFromS3 dtor 2021-03-25 23:41:03 +03:00
Azat Khuzhin
e1de960025 Fix uncaught exception in InterserverIOHTTPHandler 
There was one more uncaught exception case [1]:

    2021.03.19 18:11:00.845632 [ 17469 ] {} <Debug> InterserverIOHTTPHandler: Done processing query
    ...
    2021.03.19 18:11:31.698961 [ 80145 ] {} <Fatal> BaseDaemon: ########################################
    2021.03.19 18:11:31.699903 [ 80145 ] {} <Fatal> BaseDaemon: (version 21.4.1.6293 (official build), build id: 859E400E1C65C4702FE491420741DD8B58190002) (from thread 17469) (no query) Received signal Aborted (6)
    2021.03.19 18:11:32.614075 [ 80145 ] {} <Fatal> BaseDaemon: 8. ./obj-x86_64-linux-gnu/../contrib/libcxxabi/src/cxa_handlers.cpp:89: std::terminate() @ 0x21e9b3a2 in /usr/bin/clickhouse
    2021.03.19 18:11:43.831215 [ 80145 ] {} <Fatal> BaseDaemon: 10. ./obj-x86_64-linux-gnu/../src/Server/HTTP/WriteBufferFromHTTPServerResponse.cpp:201: ? @ 0x1be77038 in /usr/bin/clickhouse
    2021.03.19 18:11:44.743193 [ 80145 ] {} <Fatal> BaseDaemon: 11. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:892: std::__1::allocator<DB::WriteBufferFromHTTPServerResponse>::destroy(DB::WriteBufferFromHTTPServerResponse*) @ 0x1bddd7c9 in /usr/bin/clickhouse
    2021.03.19 18:11:45.283905 [ 80145 ] {} <Fatal> BaseDaemon: 12. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/__memory/allocator_traits.h:541: void std::__1::allocator_traits<std::__1::allocator<DB::WriteBufferFromHTTPServerResponse> >::__destroy<DB::WriteBufferFromHTTPServerResponse>(std::__1::integral_constant<bool, true>, std::__1::allocator<DB::WriteBufferFromHTTPServerResponse>&, DB::WriteBufferFromHTTPServerResponse*) @ 0x1bddd79d in /usr/bin/clickhouse
    2021.03.19 18:11:45.805233 [ 80145 ] {} <Fatal> BaseDaemon: 13. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/__memory/allocator_traits.h:487: void std::__1::allocator_traits<std::__1::allocator<DB::WriteBufferFromHTTPServerResponse> >::destroy<DB::WriteBufferFromHTTPServerResponse>(std::__1::allocator<DB::WriteBufferFromHTTPServerResponse>&, DB::WriteBufferFromHTTPServerResponse*) @ 0x1bddd76d in /usr/bin/clickhouse
    2021.03.19 18:11:46.351371 [ 80145 ] {} <Fatal> BaseDaemon: 14. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2611: std::__1::__shared_ptr_emplace<DB::WriteBufferFromHTTPServerResponse, std::__1::allocator<DB::WriteBufferFromHTTPServerResponse> >::__on_zero_shared() @ 0x1bddd525 in /usr/bin/clickhouse
    2021.03.19 18:11:46.579263 [ 80145 ] {} <Fatal> BaseDaemon: 15. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2476: std::__1::__shared_count::__release_shared() @ 0x119490ed in /usr/bin/clickhouse
    2021.03.19 18:11:46.790912 [ 80145 ] {} <Fatal> BaseDaemon: 16. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2517: std::__1::__shared_weak_count::__release_shared() @ 0x1194908f in /usr/bin/clickhouse
    2021.03.19 18:11:47.277990 [ 80145 ] {} <Fatal> BaseDaemon: 17. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3213: std::__1::shared_ptr<DB::WriteBufferFromHTTPServerResponse>::~shared_ptr() @ 0x1bdd75fc in /usr/bin/clickhouse
    2021.03.19 18:11:47.649213 [ 80145 ] {} <Fatal> BaseDaemon: 18. ./obj-x86_64-linux-gnu/../src/Server/InterserverIOHTTPHandler.h:34: DB::InterserverIOHTTPHandler::Output::~Output() @ 0x1bdf6bd5 in /usr/bin/clickhouse
    2021.03.19 18:11:47.921556 [ 80145 ] {} <Fatal> BaseDaemon: 19. ./obj-x86_64-linux-gnu/../src/Server/InterserverIOHTTPHandler.cpp:154: DB::InterserverIOHTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x1bdf653f in /usr/bin/clickhouse

  [1]: https://clickhouse-test-reports.s3.yandex.net/0/78c56b891383288cf3a893139e796fc87476412e/stress_test_(debug).html

Since in case of no errors during processing we should call finalize, to
ensure that it will not be called from dtor.

Fixes: #22046
Fixes: #22067
 2021-03-25 23:41:03 +03:00
Azat Khuzhin
48fe30e5a2 Add missing logging for exception in InterserverIOHTTPHandler 2021-03-25 23:41:03 +03:00
Azat Khuzhin
8e445c5e55 Disable async_socket_for_remote/use_hedged_requests for buggy linux kernels 
async_socket_for_remote/use_hedged_requests uses nested epolls, which
may not be reliable in 5.5+ [1], but it has been fixed in shortly in
5.7+ [2] or
5.6.13+.

  [1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=339ddb53d373
  [2]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0c54a6a44bf3
 2021-03-25 23:39:30 +03:00
Alexey Milovidov
a8ce138788 Speedup codec NONE 2021-03-25 23:08:47 +03:00
Alexander Kuzmenkov
33647ef3d5 fix build 2021-03-25 23:02:09 +03:00
Alexander Kuzmenkov
0675f9403c
Merge pull request #21895 from ClickHouse/aku/laglead 
float RANGE OFFSET and lag/lead_in_frame
 2021-03-25 23:00:22 +03:00
Kruglov Pavel
8c6da972da
Merge pull request #22059 from sevirov/sevirov-DOCSUP-4916-add_orc_output_format 
DOCSUP-4916: Add ORC output format
 2021-03-25 22:13:37 +03:00
Dmitriy
a4aff546e9 Fix the title 
Поправил название заголовка.
 2021-03-25 22:05:26 +03:00
Alexander Kuzmenkov
7444eab536 Merge remote-tracking branch 'origin/aku/negate-stable' into HEAD 2021-03-25 21:40:02 +03:00
Alexander Kuzmenkov
9aa89161dc fix for decimals 2021-03-25 21:39:57 +03:00
Maksim Kita
bbe1960eea
Merge pull request #21573 from kitaisreal/updated-cache-dictionary-storage 
Added specialized CacheDictionaryStorage
 2021-03-25 21:29:01 +03:00
Alexey Milovidov
177a017c5c Minor modification #22115 2021-03-25 20:51:32 +03:00
Alexander Kuzmenkov
72b742258a
Update src/Parsers/ASTFunction.cpp 2021-03-25 19:20:36 +03:00
Alexander Kuzmenkov
4e96675a30
Update src/Parsers/ASTFunction.cpp 2021-03-25 19:18:57 +03:00
Dmitriy
b179ae468c Update formats.md 
Внес небольшие правки в русскую версию.
 2021-03-25 19:11:30 +03:00
Alexander Kuzmenkov
8ea697b7df use camelCase 2021-03-25 18:49:45 +03:00
Alexander Kuzmenkov
48af3f5851 stable formatting for negate() 2021-03-25 18:38:43 +03:00
Ewout
6fe26493e9 Fix docker entrypoint in case http_port is not in config. 
`http_port` can be removed from the configuration when serving HTTP is
not desired. When it is not present, `extract-from-config` will fail,
which will prevent the server from starting.

Only fetch it when it is required for database initialization.
 2021-03-25 16:14:48 +01:00
Peng Jian
fc1cd6f65e remove unnecessary code 2021-03-25 22:57:08 +08:00
vdimir
8f5c8f32df
Shrink totals block to 1 row for JOIN with TOTALS and arrayJoin 2021-03-25 17:32:36 +03:00
Kruglov Pavel
66b8431e71
Merge pull request #21429 from Avogar/secure-socket 
Fix receive and send timeouts and non-blocking read in secure socket
 2021-03-25 17:30:17 +03:00
Anton Popov
da2d0d2682 fix build 2021-03-25 17:18:29 +03:00
Peng Jian
2ede490152 The thread pool name should not longer than 15 chars 2021-03-25 21:57:47 +08:00
michon470
d8931e504e
Merge branch 'master' into kuriatnikovm-DOCSUP-7105-translate_changes 2021-03-25 16:32:39 +03:00
feng lv
55ba4ec15e Fix bar with invalid float value 
fix
 2021-03-25 13:28:07 +00:00
alexey-milovidov
e5b96bd93b
Merge pull request #22105 from amosbird/rp 
Bump replxx
 2021-03-25 16:04:13 +03:00
alexey-milovidov
9a9581f928
Merge pull request #22006 from ClickHouse/fix-ubsan-add-month 
Fix UBSan report in addMonths
 2021-03-25 16:03:25 +03:00
alexey-milovidov
f895bc895c
Merge pull request #22011 from ClickHouse/min_max_time_system_parts_datetime64 
Expose DateTime64 minmax part index in system.parts and system.parts_columns
 2021-03-25 16:02:33 +03:00
alexey-milovidov
52396acba1
Merge pull request #22064 from ClickHouse/fix-overflow-aes 
Fix missing check in decrypt for AEAD mode
 2021-03-25 16:01:44 +03:00
alexey-milovidov
cb6926629e
Merge pull request #22099 from ClickHouse/map-populate-series-ubsan 
Fix UBSan report in mapPopulateSeries.
 2021-03-25 16:00:21 +03:00
alexey-milovidov
37a4f3ab3a
Merge pull request #22063 from ClickHouse/tuple-null-in 
Allow to search tuple of NULLs in a set
 2021-03-25 15:59:41 +03:00
Anton Popov
6a15431be7 Merge remote-tracking branch 'upstream/master' into HEAD 2021-03-25 15:57:35 +03:00
Anton Popov
f58d470dd6
Merge pull request #21830 from foolchi/optimize_select_with_final 
optimize select final with prewhere primary key
 2021-03-25 15:32:13 +03:00
alesapin
4c28c25a75 Merge branch 'master' into jepsen_for_nukeeper 2021-03-25 14:34:00 +03:00
alesapin
3d92cb46f7 Trying to fix my favorite test 2021-03-25 14:22:19 +03:00
feng lv
b7622868fc remove useless code 2021-03-25 11:10:41 +00:00
Vitaly Baranov
f6f739eca5
Merge pull request #22027 from vitlibar/external-loader-small-simplification 
Small simplification in ExternalLoader.
 2021-03-25 13:43:31 +03:00
Vitaly Baranov
86a293d6c8
Merge pull request #22057 from vitlibar/update-simdjson 
Update simdjson
 2021-03-25 13:27:05 +03:00
alesapin
640ba79288 Remove data corruption from lightweight run 2021-03-25 13:23:25 +03:00
alesapin
d36d3f036d Fix several races in NuRaft 2021-03-25 13:04:16 +03:00
benbiti
a861b633d6 fix broken links in settings.md 2021-03-25 17:09:20 +08:00
benbiti
1f34391018 update links in uuid 2021-03-25 17:09:20 +08:00
benbiti
f9c7fd7b9d update zh docs datatype uuid 2021-03-25 17:09:20 +08:00