Alexey Milovidov
3bd9a7d3bd
Add a test for #12297
2020-11-29 15:23:16 +03:00
a.palagashvili
502355922d
added errors checking, removed redundant variables
2020-11-29 15:14:42 +03:00
alexey-milovidov
00da5148a1
Merge pull request #17347 from azat/substringCount
...
Implement countSubstrings()
2020-11-29 14:32:33 +03:00
alexey-milovidov
fabceebbce
Merge pull request #17145 from amosbird/cddt
...
Fix unmatched type comparison in KeyCondition
2020-11-29 14:29:35 +03:00
alexey-milovidov
43aa7740fb
Merge pull request #17526 from ClickHouse/obfuscator-remove-some-words
...
Query obfuscator: remove some words
2020-11-29 14:25:09 +03:00
Alexey Milovidov
e635b47299
Add a test for StorageJoin and UUID
2020-11-29 10:43:34 +03:00
vdimir
5f330a9366
IP dictionary works with aliased types IPv4
and IPv6
( close #6804 )
2020-11-28 23:12:19 +03:00
alexey-milovidov
19bb345440
Merge pull request #17506 from ClickHouse/fix-bad-test
...
Fix bad test 01317_no_password_in_command_line.sh
2020-11-28 15:38:22 +03:00
Azat Khuzhin
6803e563a5
Add a test for INSERT SELECT with very small memory limits
...
I found one query, using which, pretty easy to abnormal terminate the
server, for example executing the following query:
insert into placeholder_table_name select * from numbers_mt(65535) settings max_memory_usage=1, max_untracked_memory=1 ; -- { serverError 60 }
Will lead to abnormal server termination:
22:51:36.377873 [ 1853495 ] {edae87ee-05b3-4bfa-be51-3f81d25bca2e} <Debug> executeQuery: (from [::1]:48852) insert into placeholder_table_name select * from numbers_mt(65535) format Null settings max_memory_usage=1, max_untracked_memory=1
22:51:36.390607 [ 1853446 ] {} <Trace> BaseDaemon: Received signal -1
22:51:36.390644 [ 1853446 ] {} <Fatal> BaseDaemon: (version 20.12.1.5205 (official build), build id: B23CB64BD60D7F4C) (from thread 1853495) Terminate called for uncaught exception:
e.displayText() = DB::Exception: Memory limit (for query) exceeded: would use 2.01 MiB (attempt to allocate chunk of 48 bytes), maximum: 1.00 B, Stack trace (when copying this message, always include the lines below):
uild_docker/../contrib/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x107323d0 in /src/ch/tmp/upstream/clickhouse
uild_docker/../src/Common/Exception.cpp:39: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x74f5085 in /src/ch/tmp/upstream/clickhouse
uild_docker/../contrib/libcxx/include/string:2134: DB::Exception::Exception<char const*, char const*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, long&, std::__1::basic_string<char, std::__1::char_traits<char>,
22:51:36.390667 [ 1853446 ] {} <Trace> BaseDaemon: Received signal 6
22:51:36.390801 [ 1854585 ] {} <Fatal> BaseDaemon: ########################################
22:51:36.390827 [ 1854585 ] {} <Fatal> BaseDaemon: (version 20.12.1.5205 (official build), build id: B23CB64BD60D7F4C) (from thread 1853495) (query_id: edae87ee-05b3-4bfa-be51-3f81d25bca2e) Received signal Aborted (6)
22:51:36.390842 [ 1854585 ] {} <Fatal> BaseDaemon:
22:51:36.390859 [ 1854585 ] {} <Fatal> BaseDaemon: Stack trace: 0x7ffff7dde615 0x7ffff7dc7862 0x768a37d 0x11aea23c 0x11ad6df1 0x11ad72bc 0x11ad7806 0x11af6657 0x11ae99ea 0x74df512 0x74de86c 0x74dd02c 0x11acf19c 0xd692953 0xd7bf172 0xd7bc031 0xd7b8c11 0xdf37cf9 0xdf38e9e 0x1067a1cb 0x1067a660 0x107bec66 0x107ba3d0 0x7ffff7f843e9 0x7ffff7ea1293
22:51:36.390888 [ 1854585 ] {} <Fatal> BaseDaemon: 3. raise @ 0x3d615 in /usr/lib/libc-2.32.so
22:51:36.390900 [ 1854585 ] {} <Fatal> BaseDaemon: 4. __GI_abort @ 0x26862 in /usr/lib/libc-2.32.so
22:51:36.391618 [ 1854585 ] {} <Fatal> BaseDaemon: 5. /build/build_docker/../src/IO/WriteBuffer.h:46: terminate_handler() (.cold) @ 0x768a37d in /src/ch/tmp/upstream/clickhouse
22:51:36.392363 [ 1854585 ] {} <Fatal> BaseDaemon: 6. /build/build_docker/../contrib/libcxxabi/src/cxa_handlers.cpp:61: std::__terminate(void (*)()) @ 0x11aea23c in /src/ch/tmp/upstream/clickhouse
22:51:36.393103 [ 1854585 ] {} <Fatal> BaseDaemon: 7. /build/build_docker/../contrib/libcxxabi/src/cxa_personality.cpp:325: call_terminate @ 0x11ad6df1 in /src/ch/tmp/upstream/clickhouse
22:51:36.393812 [ 1854585 ] {} <Fatal> BaseDaemon: 8. /build/build_docker/../contrib/libcxxabi/src/cxa_personality.cpp:887: scan_eh_tab @ 0x11ad72bc in /src/ch/tmp/upstream/clickhouse
22:51:36.394541 [ 1854585 ] {} <Fatal> BaseDaemon: 9. /build/build_docker/../contrib/libcxxabi/src/cxa_personality.cpp:970: __gxx_personality_v0 @ 0x11ad7806 in /src/ch/tmp/upstream/clickhouse
22:51:36.395265 [ 1854585 ] {} <Fatal> BaseDaemon: 10. /build/obj-x86_64-linux-gnu/../contrib/libunwind/src/UnwindLevel1.c:101: _Unwind_RaiseException @ 0x11af6657 in /src/ch/tmp/upstream/clickhouse
22:51:36.395979 [ 1854585 ] {} <Fatal> BaseDaemon: 11. /build/build_docker/../contrib/libcxxabi/src/cxa_exception.cpp:151: __cxa_throw @ 0x11ae99ea in /src/ch/tmp/upstream/clickhouse
22:51:36.396038 [ 1854585 ] {} <Fatal> BaseDaemon: 12. /build/build_docker/../src/Common/MemoryTracker.cpp:171: MemoryTracker::alloc(long) (.cold) @ 0x74df512 in /src/ch/tmp/upstream/clickhouse
22:51:36.396212 [ 1854585 ] {} <Fatal> BaseDaemon: 13. /build/build_docker/../src/Common/MemoryTracker.cpp:177: MemoryTracker::alloc(long) @ 0x74de86c in /src/ch/tmp/upstream/clickhouse
22:51:36.396225 [ 1854585 ] {} <Fatal> BaseDaemon: 14. /build/build_docker/../base/common/../common/memory.h:20: operator new(unsigned long) @ 0x74dd02c in /src/ch/tmp/upstream/clickhouse
22:51:36.396948 [ 1854585 ] {} <Fatal> BaseDaemon: 15. /build/build_docker/../contrib/libcxx/include/string:1495: std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::basic_string(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x11acf19c in /src/ch/tmp/upstream/clickhouse
22:51:36.397683 [ 1854585 ] {} <Fatal> BaseDaemon: 16. /build/build_docker/../contrib/libcxx/include/list:361: DB::ProcessListEntry::~ProcessListEntry() @ 0xd692953 in /src/ch/tmp/upstream/clickhouse
22:51:36.399234 [ 1854585 ] {} <Fatal> BaseDaemon: 17. /build/build_docker/../contrib/libcxx/include/memory:3483: std::__1::shared_ptr<DB::ProcessListEntry>::~shared_ptr() @ 0xd7bf172 in /src/ch/tmp/upstream/clickhouse
22:51:36.400472 [ 1854585 ] {} <Fatal> BaseDaemon: 18. /build/build_docker/../src/Interpreters/executeQuery.cpp:428: DB::executeQueryImpl(char const*, char const*, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool, DB::ReadBuffer*) (.cold) @ 0xd7bc031 in /src/ch/tmp/upstream/clickhouse
22:51:36.401525 [ 1854585 ] {} <Fatal> BaseDaemon: 19. /build/build_docker/../src/Interpreters/executeQuery.cpp:814: DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) @ 0xd7b8c11 in /src/ch/tmp/upstream/clickhouse
22:51:36.402887 [ 1854585 ] {} <Fatal> BaseDaemon: 20. /build/build_docker/../src/Server/TCPHandler.cpp:254: DB::TCPHandler::runImpl() @ 0xdf37cf9 in /src/ch/tmp/upstream/clickhouse
22:51:36.404453 [ 1854585 ] {} <Fatal> BaseDaemon: 21. /build/build_docker/../src/Server/TCPHandler.cpp:1336: DB::TCPHandler::run() @ 0xdf38e9e in /src/ch/tmp/upstream/clickhouse
22:51:36.405158 [ 1854585 ] {} <Fatal> BaseDaemon: 22. /build/build_docker/../contrib/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x1067a1cb in /src/ch/tmp/upstream/clickhouse
22:51:36.405918 [ 1854585 ] {} <Fatal> BaseDaemon: 23. /build/build_docker/../contrib/libcxx/include/atomic:856: Poco::Net::TCPServerDispatcher::run() @ 0x1067a660 in /src/ch/tmp/upstream/clickhouse
22:51:36.406690 [ 1854585 ] {} <Fatal> BaseDaemon: 24. /build/build_docker/../contrib/poco/Foundation/include/Poco/Mutex_POSIX.h:59: Poco::PooledThread::run() @ 0x107bec66 in /src/ch/tmp/upstream/clickhouse
22:51:36.407413 [ 1854585 ] {} <Fatal> BaseDaemon: 25. /build/build_docker/../contrib/poco/Foundation/include/Poco/AutoPtr.h:215: Poco::ThreadImpl::runnableEntry(void*) @ 0x107ba3d0 in /src/ch/tmp/upstream/clickhouse
22:51:36.407428 [ 1854585 ] {} <Fatal> BaseDaemon: 26. start_thread @ 0x93e9 in /usr/lib/libpthread-2.32.so
22:51:36.407436 [ 1854585 ] {} <Fatal> BaseDaemon: 27. clone @ 0x100293 in /usr/lib/libc-2.32.so
But after other patches in this series it wont.
2020-11-28 11:15:26 +03:00
Alexey Milovidov
7c0138542b
Query obfuscator: remove some words
2020-11-28 10:55:15 +03:00
alesapin
1d179dac22
Merge pull request #17486 from ClickHouse/fix_more_flaky_tests
...
Fix more flaky integration tests
2020-11-28 10:43:29 +03:00
alexey-milovidov
c189f6405f
Merge pull request #16767 from azat/optimize_trivial_count_query-fix
...
Fix optimize_trivial_count_query with partition predicate
2020-11-28 08:43:12 +03:00
alexey-milovidov
62a45e8ff7
Merge pull request #17414 from azat/fix-INSERT-SETTINGS-parsing
...
Fix parsing of SETTINGS clause of the INSERT ... SELECT ... SETTINGS query
2020-11-28 08:38:42 +03:00
alexey-milovidov
61a8406c1e
Merge pull request #17485 from ClickHouse/aku/perf-thresholds
...
Adjust perf test thresholds
2020-11-28 08:34:44 +03:00
alexey-milovidov
7a6c72ce88
Merge pull request #17471 from CurtizJ/fix-order-by-functions
...
Fix 'optimize_redundant_functions_in_order_by'
2020-11-28 08:34:26 +03:00
Alexey Milovidov
44b059c681
Fix bad test 01317_no_password_in_command_line.sh
2020-11-28 08:27:59 +03:00
Nikita Mikhaylov
6f3db7ff50
Merge pull request #17330 from CurtizJ/move-to-prewhere
...
Allow to move conditions to prewhere with compact parts
2020-11-28 02:02:42 +03:00
tavplubix
aaaa7ad59f
Merge pull request #16806 from TCeason/add_test_for_materialize_mysql
...
add network partition integration test for MaterializeMySQL
2020-11-27 20:39:06 +03:00
alesapin
6567796014
Fix kill mutation on concurrent alter queries
2020-11-27 18:46:52 +03:00
Anton Popov
458395dcaf
fix test
2020-11-27 17:31:30 +03:00
Nikita Mikhaylov
0b6f5c75b9
Merge pull request #17435 from vitlibar/grpc-protocol-2
...
Implement GRPC protocol (corrections)
2020-11-27 15:19:50 +03:00
Alexander Kuzmenkov
a3277b183d
Adjust perf test thresholds
2020-11-27 15:08:42 +03:00
alesapin
d199a2bf76
Merge branch 'master' into complete_zk_api
2020-11-27 14:55:55 +03:00
alesapin
17fa29f44f
Less pulls of network image
2020-11-27 14:38:04 +03:00
Anton Popov
601d633690
fix optimize_redundant_functions_in_order_by
2020-11-27 14:29:39 +03:00
alesapin
f12ba9f2c1
Fix test
2020-11-27 14:26:42 +03:00
Alexander Kuzmenkov
19bdb34b19
Merge pull request #16183 from hexiaoting/dev_replace
...
Fix: throw error when column transformer use non-exist column
2020-11-27 14:09:58 +03:00
alesapin
50cf7fda39
Less flaky test_realod_zookeeper
2020-11-27 14:05:59 +03:00
Nikolai Kochetov
f3bea2210c
Update tests.
2020-11-27 13:55:45 +03:00
Nikolai Kochetov
85283b3944
Merge pull request #17397 from ClickHouse/fix-types-for-read-from-join
...
Fix crash while reading from JOIN table with LowCardinality types
2020-11-27 13:37:05 +03:00
Nikolai Kochetov
60af8219ee
Merge pull request #17439 from ClickHouse/chengxianglibra-dist-agg-issue
...
Fix incorrect optimization of distinct
2020-11-27 13:26:19 +03:00
Nikolai Kochetov
c206b2506b
Update tests.
2020-11-27 13:10:09 +03:00
alexey-milovidov
dfae1efbbd
Merge pull request #17070 from fastio/master
...
Support multiple ZooKeeper clusters
2020-11-27 10:38:01 +03:00
taichong
c8f7a56a25
fix CI timeout err
2020-11-27 11:34:49 +08:00
hexiaoting
2523f8f0c6
Merge remote-tracking branch 'origin/master' into dev_replace
2020-11-27 10:51:04 +08:00
alexey-milovidov
572bdb4090
Merge pull request #17423 from ClickHouse/remove-time-series-group-sum
...
Remove timeSeriesGroupSum
2020-11-26 23:07:46 +03:00
Azat Khuzhin
838596c7a4
Implement countSubstrings()
...
Function to count number of substring occurrences in the string:
- in case of needle is multi char - counts non-intersecting substrings
- the code is based on position helpers.
The following new functions is available:
- countSubstrings()
- countSubstringsCaseInsensitive()
- countSubstringsCaseInsensitiveUTF8()
v0: substringCount()
v2:
- add substringCountCaseInsensitiveUTF8
- improve tests
- fix coding style issues
- fix multichar needle
v3: rename to countSubstrings (by analogy with countEqual())
2020-11-26 22:58:16 +03:00
alexey-milovidov
1cd09fa367
Merge pull request #15127 from ucasFL/add-mutation-for-storagememory
...
Add mutation support for StorageMemory
2020-11-26 22:22:59 +03:00
Nikolai Kochetov
9291bbb04b
Merge pull request #16804 from vdimir/ip-dict-no-trie
...
sorted-array based ip_dict
2020-11-26 19:26:06 +03:00
Alexander Kuzmenkov
15a0f14445
Merge pull request #15419 from myrrc/improvement/diff-types-in-avg-weighted
...
Allow different types in avgWeighted. Allow avg and avgWeighed to operate on extended integral types.
2020-11-26 17:16:48 +03:00
alesapin
90bddce5ed
Merge pull request #17410 from ClickHouse/test_mutation_for_empty_partitions
...
Add test for mutation with empty partition
2020-11-26 16:42:43 +03:00
Nikita Mikhaylov
084c75fa6e
Merge pull request #17309 from nikitamikhaylov/merging-sequential-consistency
...
Merging #16309
2020-11-26 15:26:05 +03:00
Nikita Mikhaylov
adb82649c4
Merge pull request #16883 from nikitamikhaylov/refactor-rank-corr
...
mannWhitneyUTest + studentTTest + welchTTest + small rankCorr refactor
2020-11-26 15:17:18 +03:00
Nikolai Kochetov
83f2fc3d9c
Update test.
2020-11-26 12:51:55 +03:00
Nikolai Kochetov
42399e8866
Merge branch 'dist-agg-issue' of https://github.com/chengxianglibra/ClickHouse into chengxianglibra-dist-agg-issue
2020-11-26 11:35:47 +03:00
Amos Bird
022ba2b0a9
Fix unmatched type comparison in KeyCondition
2020-11-26 16:15:50 +08:00
alexey-milovidov
ee3a0b790b
Merge pull request #17376 from azat/toUnixTimestamp-Date-fix
...
Prohibit toUnixTimestamp(Date())
2020-11-26 10:50:00 +03:00
Vitaly Baranov
1fc43b3c93
Remove trailing whitespaces in config.xml
2020-11-26 10:27:04 +03:00
Vitaly Baranov
49cf980761
Use port 9100 for grpc by default.
2020-11-26 10:14:27 +03:00
alexey-milovidov
747453b008
Merge pull request #17051 from hexiaoting/bugfix_fuzzbits
...
Bug fix for funciton fuzzBits
2020-11-26 06:41:20 +03:00
Alexey Milovidov
85d3e62275
Remove timeSeriesGroupSum
2020-11-26 05:46:28 +03:00
alexey-milovidov
fb3a69b298
Merge pull request #17254 from azat/fix-dist-query-cancelation
...
Fix "Unexpected packet Data received from client" for Distributed queries
2020-11-26 03:46:27 +03:00
alexey-milovidov
c438b43892
Merge pull request #17375 from ClickHouse/always-working-test
...
Added a test for what was always working
2020-11-26 03:29:14 +03:00
Nikita Mikhaylov
ede023a237
rerun tests to be sure
2020-11-26 01:24:42 +03:00
Alexander Kuzmenkov
394b81ac46
Merge remote-tracking branch 'origin/master' into HEAD
2020-11-25 23:38:55 +03:00
Azat Khuzhin
0b47f4a9e9
Fix optimize_trivial_count_query with partition predicate
...
Consider the following example:
CREATE TABLE test(p DateTime, k int) ENGINE MergeTree PARTITION BY toDate(p) ORDER BY k;
INSERT INTO test VALUES ('2020-09-01 00:01:02', 1), ('2020-09-01 20:01:03', 2), ('2020-09-02 00:01:03', 3);
- SELECT count() FROM test WHERE toDate(p) >= '2020-09-01' AND p <= '2020-09-01 00:00:00'
In this case rpn will be (FUNCTION_IN_RANGE, FUNCTION_UNKNOWN (due to strict), FUNCTION_AND)
and for optimize_trivial_count_query we cannot use index if there is at least one FUNCTION_UNKNOWN.
since there is no post processing and return count() based on only the first predicate is wrong.
Before this patch FUNCTION_UNKNOWN was allowed for optimize_trivial_count_query, and the result was wrong.
And two examples above just to show the difference, the behaviour hadn't been changed with this patch:
- SELECT * FROM test WHERE toDate(p) >= '2020-09-01' AND p <= '2020-09-01 00:00:00'
In this case will be (FUNCTION_IN_RANGE, FUNCTION_IN_RANGE (due to non-strict), FUNCTION_AND)
so it will prune everything out and nothing will be read.
- SELECT * FROM test WHERE toDate(p) >= '2020-09-01' AND toUnixTimestamp(p)%5==0
In this case will be (FUNCTION_IN_RANGE, FUNCTION_UNKNOWN, FUNCTION_AND)
and all, two, partitions will be scanned, but due to filtering later none of rows will be matched.
2020-11-25 23:09:17 +03:00
Azat Khuzhin
32f69dc76f
Cleanup 01505_trivial_count_with_partition_predicate
2020-11-25 23:09:17 +03:00
Nikolai Kochetov
729272391f
Merge branch 'master' into ip-dict-no-trie
2020-11-25 23:07:19 +03:00
Nikolai Kochetov
99f1151236
Update arcadia_skip_list.txt
2020-11-25 23:06:37 +03:00
Azat Khuzhin
3f67e320dd
Fix parsing of SETTINGS clause of the INSERT ... SELECT ... SETTINGS query
...
Before this patch the following query ignores the settings for INSERT:
insert into test_parallel_insert select * from numbers_mt(65535*2) settings max_insert_threads=10
And the reason is that SETTINGS was parsed by the SELECT parser.
Fix this by push down the SETTINGS from the SELECT to INSERT.
Also note that since INSERT parser does not use ParserQueryWithOutput the
following works:
insert into test_parallel_insert select * from numbers_mt(65535*2) format Null settings max_insert_threads=10
2020-11-25 22:53:58 +03:00
Azat Khuzhin
688cb6b4d9
Update date_time_short perf test for toUnixTimestamp(Date())
2020-11-25 21:17:11 +03:00
nikitamikhaylov
44f4064e9b
update test
2020-11-25 20:51:07 +03:00
nikitamikhaylov
02a80230f8
remove comments
2020-11-25 20:51:07 +03:00
nikitamikhaylov
71d6126c6b
better
2020-11-25 20:51:07 +03:00
nikitamikhaylov
4246971961
fix tests
2020-11-25 20:51:07 +03:00
nikitamikhaylov
de9dc3a43f
update copy pasted test
2020-11-25 20:51:06 +03:00
nikitamikhaylov
f764332070
better merge
2020-11-25 20:51:06 +03:00
nikitamikhaylov
3f874af323
new interface for the function
...
(cherry picked from commit 89547e77cf
)
2020-11-25 20:51:06 +03:00
nikitamikhaylov
9177ba3c02
test added
2020-11-25 20:51:06 +03:00
nikitamikhaylov
13081370ba
mann-whitney
2020-11-25 20:51:06 +03:00
nikitamikhaylov
9afe579225
small refactor
2020-11-25 20:51:06 +03:00
alesapin
68c8219edf
Add test for mutation with empty partition
2020-11-25 18:13:27 +03:00
alesapin
dd4c5bdeb9
Merge pull request #17249 from amosbird/insub
...
Fix const column in set index.
2020-11-25 17:05:23 +03:00
Nikolai Kochetov
8e5f7acd07
Added test.
2020-11-25 16:34:07 +03:00
alesapin
fb86eaf6fc
Better errors and better config
2020-11-25 16:19:09 +03:00
Nikita Mikhaylov
27acf6462f
Merge pull request #17248 from kaka11chen/15235
...
Fix #15235 . When clickhouse-copier handle non-partitioned table, throws segfault error.
2020-11-25 14:31:59 +03:00
Alexander Kuzmenkov
edce1e636e
Merge pull request #16535 from ClickHouse/aku/opentelemetry
...
OpenTelemetry improvements
2020-11-25 14:10:17 +03:00
myrrc
1a9c313529
fix test reference
2020-11-25 12:19:19 +03:00
alesapin
4d40d9be3f
Merge branch 'master' into complete_zk_api
2020-11-25 10:31:16 +03:00
alesapin
3e1b2f515b
Merge pull request #17311 from ClickHouse/fix_integration_tests
...
Fix some flaky tests
2020-11-25 10:09:27 +03:00
Vitaly Baranov
08b3707842
Merge pull request #15111 from vitlibar/grpc-protocol
...
Implement GRPC protocol.
2020-11-25 09:00:32 +03:00
Azat Khuzhin
9649b96bbb
Update expectations for toUnixTimestamp(Date()) in 00921_datetime64_compatibility
2020-11-25 07:42:36 +03:00
chenqi
302cd55f45
Fix #15235 . When clickhouse-copier handle non-partitioned table, throw segfault error.
2020-11-25 11:52:00 +08:00
Azat Khuzhin
17f6c82ffa
Prohibit toUnixTimestamp(Date())
...
Making it implicitly cast to Date() does not looks correct, since before
it returns somewhat unexpected result:
SELECT toUnixTimestamp(today())
┌─toUnixTimestamp(today())─┐
│ 18591 │
└──────────────────────────┘
2020-11-25 00:26:14 +03:00
Alexey Milovidov
99073c26ee
Added a test for what was always working
2020-11-25 00:00:55 +03:00
Alexander Kuzmenkov
1d7dcef4a0
Merge remote-tracking branch 'origin/master' into HEAD
2020-11-24 21:45:10 +03:00
Alexander Kuzmenkov
1d6558f694
not sure what's going on
2020-11-24 21:45:01 +03:00
Anton Popov
f2106f9683
add test
2020-11-24 20:41:11 +03:00
Vitaly Baranov
65719da0d8
Merge pull request #16977 from MyroTk/rbac_attach_create_drop_detach_testflows
...
RBAC Testflows tests for ATTACH, CREATE, DROP, and DETACH
2020-11-24 18:31:44 +03:00
Vitaly Baranov
4b3488934c
Disable running test test_odbc_interaction.test_bridge_dies_with_parent with asan.
2020-11-24 17:55:02 +03:00
Vitaly Baranov
3c85165e60
Redirect stderr, stdout for clickhouse-odbc-bridge to files.
2020-11-24 17:55:02 +03:00
Vitaly Baranov
950bccb130
Add test for authentication.
2020-11-24 17:55:02 +03:00
Vitaly Baranov
a0e384b0c0
Added support for external tables.
2020-11-24 17:55:02 +03:00
Vitaly Baranov
23842e7ac6
Added support for input() function.
2020-11-24 17:55:02 +03:00
Vitaly Baranov
0e3a8840b5
Support cancellation of executing query via gRPC.
2020-11-24 17:55:02 +03:00
Vitaly Baranov
88f1eeec2f
Add tests for parallel executing of queries.
2020-11-24 17:55:02 +03:00
Vitaly Baranov
eab3006a4d
Support SSL and compression when using gRPC protocol.
2020-11-24 17:55:02 +03:00
Vitaly Baranov
b0cb3eb306
Extend the protocol with streaming and nonstreaming functions.
2020-11-24 17:55:01 +03:00
Vitaly Baranov
b51e14253d
Added support for sessions in gRPC protocol.
2020-11-24 17:55:01 +03:00
Vitaly Baranov
9285f7edc1
Send profile info to client.
2020-11-24 17:55:01 +03:00
Vitaly Baranov
98e2cc4117
Fix sending progress.
2020-11-24 17:55:01 +03:00
Vitaly Baranov
4f0405af93
Send logs via gRPC protocol too.
2020-11-24 17:55:01 +03:00
Vitaly Baranov
218d9ea3e8
Added input_data_delimiter to protocol.
2020-11-24 17:55:01 +03:00
Vitaly Baranov
797c84889f
Use ReadBuffer to join input data from multiple query infos.
...
That fixes inserting of a row splitted between two query infos.
2020-11-24 17:55:01 +03:00
Vitaly Baranov
63c8d8124a
Added test for inserting to columns with defaults.
2020-11-24 17:55:01 +03:00
Vitaly Baranov
de4586739a
Fix using output format.
2020-11-24 17:55:01 +03:00
Vitaly Baranov
8146093794
Provide more information about errors.
2020-11-24 17:55:01 +03:00
Vitaly Baranov
a327f24e3c
Refine the protocol. Code cleanup in tests.
2020-11-24 17:53:34 +03:00
Vitaly Baranov
db9f762e73
Fix tests.
2020-11-24 17:53:34 +03:00
Vitaly Baranov
13f2352c57
Move files to right places.
2020-11-24 17:53:34 +03:00
mnkonkova
6cd1557d67
Implement GRPC protocol.
2020-11-24 17:53:34 +03:00
Amos Bird
6b4512809c
Fix const column in set index.
2020-11-24 22:31:59 +08:00
Alexander Kuzmenkov
9b0368ffa5
Merge pull request #16338 from ucasFL/union-distinct
...
Add UNION DISTINCT and setting `union_default_mode`
2020-11-24 17:30:24 +03:00
myrrc
03139c2c1e
Merge branch 'improvement/diff-types-in-avg-weighted' of github:myrrc/clickhouse into improvement/diff-types-in-avg-weighted
2020-11-24 17:08:38 +03:00
myrrc
420f2489a7
fixed decimal scales calc, updated the tests
2020-11-24 17:07:59 +03:00
Anton Popov
4e07681444
remove outdated test
2020-11-24 16:30:52 +03:00
myrrc
fbb0e6e6aa
Merge remote-tracking branch 'upstream/master' into improvement/diff-types-in-avg-weighted
2020-11-24 16:04:17 +03:00
Anton Popov
6e1a9240e1
Merge branch 'master' into fix-groupby-optimization
2020-11-24 15:24:07 +03:00
Anton Popov
ff36dee2ec
move test to bugs
2020-11-24 15:22:13 +03:00
Nikita Mikhaylov
6787f98476
Merge pull request #14958 from nikitamikhaylov/refactor_cache_dictionary
...
Refactor cache dictionary
2020-11-24 14:55:02 +03:00
nikitamikhaylov
5be300ff22
Merge branch 'master' of github.com:ClickHouse/ClickHouse into merging-sequential-consistency
2020-11-24 14:46:57 +03:00
alesapin
c1a7e4f5fa
Merge branch 'master' into complete_zk_api
2020-11-24 13:37:51 +03:00
alesapin
45f09fe293
Merge pull request #17220 from ianton-ru/s3_max_redirects
...
S3 max redirects
2020-11-24 12:13:10 +03:00
vdimir
52bc290616
Regenerate ya.make, add format null to ip_trie.xml
2020-11-24 11:20:11 +03:00
alesapin
b05385cec0
Remove code from test which copy-pasted from another test
2020-11-24 11:17:37 +03:00
alesapin
cb234e28ea
Fix garbage test
2020-11-24 11:09:10 +03:00
alesapin
5477984856
Longer integration tests for rabbitMQ
2020-11-24 10:33:35 +03:00
taichong
b5440882cd
remove kill server while insert
2020-11-24 10:11:04 +08:00
Anton Popov
2fb8717dfd
Merge pull request #17253 from amosbird/globalinfix
...
global in set index.
2020-11-24 03:06:00 +03:00
alesapin
db1aef8994
More retries
2020-11-23 23:44:46 +03:00
alesapin
3d177d414a
Merge branch 'master' into fix_integration_tests
2020-11-23 23:27:46 +03:00
Anton Popov
852a08eacb
allow to move conditions to prewhere with compact parts
2020-11-23 21:27:59 +03:00
Alexander Kuzmenkov
234bd09044
debug an unreproducible test failure
2020-11-23 20:48:00 +03:00
alexey-milovidov
a2f947ddd5
Merge pull request #16993 from azat/merge-distributed-join-fix
...
Fix Merge(Distributed()) with JOIN
2020-11-23 20:40:21 +03:00
alexey-milovidov
ac710ff0b7
Merge pull request #17299 from Jokser/flaky-s3-test-custom-auth-headers
...
Fixed flaky test_storage_s3::test_custom_auth_headers
2020-11-23 20:33:48 +03:00
Alexander Kuzmenkov
412c2fe079
Merge remote-tracking branch 'origin/master' into HEAD
2020-11-23 19:51:18 +03:00
alesapin
1c5da1c178
Better timeouts
2020-11-23 19:39:57 +03:00
alesapin
eb9b3e6737
Less global queries in functional tests
2020-11-23 17:57:46 +03:00
alesapin
c9b382b4af
Fix flaky integration tests
2020-11-23 17:40:32 +03:00
nikitamikhaylov
065cd00257
better
2020-11-23 17:24:32 +03:00
Amos Bird
172b7e9ed1
global in set index.
2020-11-23 22:05:08 +08:00
feng lv
005f345139
Merge branch 'master' of github.com:ClickHouse/ClickHouse into add-mutation-for-storagememory
2020-11-23 13:56:19 +00:00
nikitamikhaylov
68bef22fda
Merge branch 'master' of github.com:ClickHouse/ClickHouse into merging-sequential-consistency
2020-11-23 16:28:35 +03:00
tavplubix
7af89cba8a
Merge pull request #16715 from zhang2014/fix/ISSUES-16605
...
ISSUES-16605 fix without returning affected rows when insert select query in MySQL handler
2020-11-23 15:54:43 +03:00
Anton Ivashkin
aa208a9d9f
Use only 's3_max_redirect' in params instead of all settings
2020-11-23 14:02:17 +03:00
Pavel Kovalenko
5f98d38689
Fixed flaky test_storage_s3::test_custom_auth_headers
2020-11-23 13:19:43 +03:00
taichong
a6c2a90756
modify integration test materialize mysql database
...
1. fix asan test err
2. reset pymysql conn if ping err
3. finish mysql & ck killed test
4. modify check_query
2020-11-23 17:55:07 +08:00
alexey-milovidov
f38db2d421
Update 01560_merge_distributed_join.sql
2020-11-23 12:51:59 +03:00
Alexey Milovidov
9969a73d50
Merge branch 'master' into azat-merge-distributed-join-fix
2020-11-23 12:43:15 +03:00
Alexey Milovidov
ed679d8edb
Better test
2020-11-23 12:38:03 +03:00
Alexey Milovidov
24710046c3
Arcadia does not support distributed tests
2020-11-23 12:34:23 +03:00
Alexey Milovidov
b10283f8d5
Add a test for #13990
2020-11-23 12:32:57 +03:00
lichengxiang
fdde2d1e8a
fix #17294 : Distinct on subquery with group by may return duplicate result
2020-11-23 17:06:59 +08:00
alesapin
b8fb44a7b9
Merge with master
2020-11-23 11:05:50 +03:00
taichong
f99d6ea799
add ck & mysql killed while insert
2020-11-23 15:47:44 +08:00
taichong
122740767f
add integration test err
2020-11-23 15:47:44 +08:00
taichong
714767f491
make it normal
2020-11-23 15:47:44 +08:00
taichong
5a50535abc
add mysql kill sync id test
2020-11-23 15:47:44 +08:00
taichong
81bdf50085
add some except error test
2020-11-23 15:47:44 +08:00
taichong
e691b4a0d4
add network partition integration test for MaterializeMySQL
2020-11-23 15:47:44 +08:00
taichong
e8628f61e0
add mysql kill sync id test
2020-11-23 15:47:44 +08:00
taichong
3287f691ee
add some except error test
2020-11-23 15:47:44 +08:00
taichong
5c3d4b2245
add network partition integration test for MaterializeMySQL
2020-11-23 15:47:44 +08:00
feng lv
18491d8920
fix
2020-11-23 05:30:36 +00:00
Alexey Milovidov
fbbf439338
Merge branch 'master' into Enmk-DateTime64_fixes_comparison
2020-11-23 05:46:17 +03:00
alexey-milovidov
9ed0c3c85a
Merge pull request #17238 from azat/no-log-rotate-in-tests
...
Disable rotation of the logs in tests (on CI)
2020-11-23 05:40:58 +03:00
alesapin
9c8b0da382
Merge pull request #16033 from nvartolomei/nv/parts-uuid
...
Add unique identifiers IMergeTreeDataPart structure
2020-11-22 16:13:19 +03:00
zhang2014
e1e5eede86
ISSUES-16605 try fix integration failure
2020-11-22 20:13:56 +08:00
zhang2014
5b3154b298
ISSUES-16605 try fix integration test failure
2020-11-22 20:13:56 +08:00
zhang2014
a6dbba5932
ISSUES-16605 add integration test
2020-11-22 20:13:56 +08:00
alexey-milovidov
870a56b962
Merge pull request #17255 from azat/perf-tests-settings-cleanup
...
Cleanup settings handling in perf tests
2020-11-21 22:59:44 +03:00
alexey-milovidov
1663b13215
Merge pull request #17257 from ucasFL/fix
...
fix toUInt256 stack overflow
2020-11-21 22:54:06 +03:00
fenglv
268b15f221
fix toUInt256 stack overflow
2020-11-21 12:59:40 +00:00
Azat Khuzhin
8931d3eb6f
Do not use SET via <full_query> in perf tests
...
Since if the connection will be closed (by some reason), then the
setting will not be applied after transparent reconnect (since only
native clickhouse-client can do this, since it parses the query, but
perf tests uses python driver).
Just use inplace SETTINGS clause or <settings>.
2020-11-21 14:02:21 +03:00
Azat Khuzhin
ee4267c1ba
Rename 01247_optimize_distributed_group_by_sharding_key to distingiush it
...
Otherwise flaky tests will run then both, not a problem, but this is
just to trigger flaky check, since there is some tricky issue [1]:
2020-11-20 00:35:51 01247_optimize_distributed_group_by_sharding_key: [ FAIL ] 0.67 sec. - return code 101
2020-11-20 00:35:51 Received exception from server (version 20.12.1):
2020-11-20 00:35:51 Code: 101. DB::Exception: Received from localhost:9000. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: Unexpected packet Data received from client.
[1]: https://clickhouse-test-reports.s3.yandex.net/16996/8d71564056925df1415880f382aaa169cbdf37b0/functional_stateless_tests_flaky_check_(address)/test_run.txt.out.log
2020-11-21 13:32:17 +03:00
Azat Khuzhin
a3116d5614
Tune aggregating_merge_tree_simple_aggregate_function_string to make it faster
2020-11-21 12:08:59 +03:00
Nikolai Kochetov
bac1def5f9
Merge pull request #17134 from abyss7/tcp-port
...
Implement tcpPort() function for tests
2020-11-20 20:32:55 +03:00
Pavel Kruglov
ca3fe49a2a
Make setting global
2020-11-20 17:29:13 +03:00
Nicolae Vartolomei
7c8bc1c04e
Use JSON metadata in WAL
2020-11-20 13:49:17 +00:00
Nicolae Vartolomei
94293ca3ce
Assign UUIDs to parts only when configured to do so
...
Avoid breaking backwards compatibility by default for now.
2020-11-20 13:49:17 +00:00
tavplubix
3e335c1e95
Merge pull request #17091 from zhang2014/fix/binlog_checksum
...
Fix MaterializeMySQL SYNC failure when modify binlog_checksum
2020-11-20 13:25:14 +03:00
Alexander Kuzmenkov
b05ac888bd
Merge remote-tracking branch 'origin/master' into HEAD
2020-11-20 12:50:08 +03:00
Alexander Kuzmenkov
bde805cb3e
Merge pull request #17175 from ClickHouse/aku/named-tuple
...
Allow formatting named tuples as JSON objects
2020-11-20 12:48:24 +03:00
Nikolai Kochetov
31886fdca6
Merge pull request #16811 from amosbird/pf2
...
Fix verbatim partition pruner.
2020-11-20 12:07:33 +03:00
alesapin
9920d77a93
Merge branch 'master' into complete_zk_api
2020-11-20 11:48:12 +03:00
Anton Ivashkin
a7d4f4be65
Add 's3_max_redirects' test
2020-11-20 11:18:44 +03:00
alesapin
bd5d2d89ad
Merge pull request #17201 from ClickHouse/fix_read_in_order_test
...
Trying to make read_in_order_many_parts more stable
2020-11-20 10:10:17 +03:00
Kruglov Pavel
51bcd286f2
Merge pull request #16845 from azat/cutToFirstSignificantSubdomainWithWWW
...
Add cutToFirstSignificantSubdomainWithWWW()
2020-11-20 01:44:57 +03:00
Azat Khuzhin
35231662b3
Improve performance of AggregatingMergeTree w/ SimpleAggregateFunction(String)
...
While reading from AggregatingMergeTree with
SimpleAggregateFunction(String) in primary key and
optimize_aggregation_in_order perf top shows:
Samples: 1M of event 'cycles', 4000 Hz, Event count (approx.): 287759760270 lost: 0/0 drop: 0/0
Children Self Shared Object Symbol
+ 12.64% 11.39% clickhouse [.] memcpy
+ 9.08% 0.23% [unknown] [.] 0000000000000000
+ 8.45% 8.40% clickhouse [.] ProfileEvents::increment # <-- this, and in debug it has not 0.08x overhead, but 5.8x overhead
+ 7.68% 7.67% clickhouse [.] LZ4_compress_fast_extState
+ 5.29% 5.22% clickhouse [.] DB::IAggregateFunctionHelper<DB::AggregateFunctionNullUnary<true, true> >::addFree
The reason is obvious, ProfileEvents is atomic counters (and also they
are nested):
<details>
```
Samples: 7M of event 'cycles', 4000 Hz, Event count (approx.): 450726149337
ProfileEvents::increment /usr/bin/clickhouse [Percent: local period]
Percent│
│
│
│ Disassembly of section .text:
│
│ 00000000078d8900 <ProfileEvents::increment(unsigned long, unsigned long)@@Base>:
│ ProfileEvents::increment(unsigned long, unsigned long):
0.17 │ push %rbp
0.00 │ mov %rsi,%rbp
0.04 │ push %rbx
0.20 │ mov %rdi,%rbx
0.17 │ sub $0x8,%rsp
0.26 │ → callq DB::CurrentThread::getProfileEvents
│ ProfileEvents::Counters::increment(unsigned long, unsigned long):
0.00 │ lea 0x0(,%rbx,8),%rdi
0.05 │ nop
│ unsigned long std::__1::__cxx_atomic_fetch_add<unsigned long, unsigned long>(std::__1::__cxx_atomic_base_impl<unsigned long>*, unsigned long, std::__1::memory_order):
1.02 │ mov (%rax),%rdx
97.04 │ lock add %rbp,(%rdx,%rdi,1)
│ ProfileEvents::Counters::increment(unsigned long, unsigned long):
0.21 │ mov 0x10(%rax),%rax
0.04 │ test %rax,%rax
0.00 │ → jne 78d8920 <ProfileEvents::increment(unsigned long, unsigned long)@@Base+0x20>
│ ProfileEvents::increment(unsigned long, unsigned long):
0.38 │ add $0x8,%rsp
0.00 │ pop %rbx
0.04 │ pop %rbp
0.38 │ ← retq
```
</details>
These ProfileEvents was ArenaAllocChunks (it shows ~1.5M events per
second), and the reason is that the table has
SimpleAggregateFunction(String) in PK, which requires Arena.
But most of the time there Arena wasn't even used, so avoid this cost by
re-creating Arena only if it was "used" (i.e. has new chunks).
Another possibility is to avoid populating Arena::head in ctor, but this
will make the Arena code more complex, so for now this was preferred.
Also as a long-term solution it worth looking at implementing them via
RCU (to move the extra overhead out from the write code path into read
side).
2020-11-19 23:06:12 +03:00
Alexander Kuzmenkov
87a306ed23
Enable OpenTelemetry tracing in functional tests
2020-11-19 18:58:33 +03:00
vdimir
36544a45b7
Merge remote-tracking branch 'upstream/master' into ip-dict-no-trie
2020-11-19 18:56:24 +03:00
alesapin
2623d35f68
Merge pull request #17120 from ClickHouse/fix_granularity_on_block_borders
...
Fix index granularity calculation on block borders
2020-11-19 18:36:07 +03:00
Nicolae Vartolomei
425dc4b11b
Add unique identifiers IMergeTreeDataPart structure
...
For now uuids are not generated at all, they are present only if the
part is updated manually (as you can see in the integration test).
The only place where they can be seen today by an end user is in
`system.parts` table. I was looking for hiding this column behind an
option but couldn't find an easy way to do that.
Likely this is also required for WAL, but need to think how not to break
compatibility.
Relates to #13574 , https://github.com/ClickHouse/ClickHouse/issues/13574
Next 1: In the upcoming PR the plan is to integrate de-duplication based on
these fingerprints in the query pipeline.
Next 2: We'll enable automatic generation of uuids and come up with a
way for conditionally sending uuids when processing distributed queries
only when part movement is in progress.
2020-11-19 13:14:25 +00:00
Alexander Kuzmenkov
f326536ef0
fixup
2020-11-19 15:30:44 +03:00
Alexander Kuzmenkov
bcf15170ed
Merge remote-tracking branch 'origin/master' into HEAD
2020-11-19 15:29:51 +03:00
Alexander Kuzmenkov
24293ccb30
Merge remote-tracking branch 'origin/master' into HEAD
2020-11-19 15:28:37 +03:00
Alexander Kuzmenkov
f2b4c15ab0
Merge pull request #17152 from ClickHouse/aku/json-array
...
Write rows as JSON array in JSONEachRow output format
2020-11-19 15:27:31 +03:00
alesapin
1aba61be7f
Merge branch 'master' into complete_zk_api
2020-11-19 13:32:15 +03:00
alesapin
cdceafdd89
Trying to make read_in_order_many_parts more stable
2020-11-19 13:25:39 +03:00
tavplubix
8b6fbea199
trigger CI
2020-11-19 12:47:24 +03:00