Vasily Nemkov
168155eeec
Minor: cleanup
2020-12-07 18:07:40 +03:00
Vasily Nemkov
f01a566646
Updated tests
2020-12-07 17:42:49 +03:00
Vasily Nemkov
70ea507dae
OPTIMIZE DEDUPLICATE BY columns
...
Extended OPTIMIZE ... DEDUPLICATE syntax to allow explicit (or implicit with asterisk/column transformers) list of columns to check for duplicates on.
Following syntax variants are now supported:
OPTIMIZE TABLE table DEDUPLICATE; -- the old one
OPTIMIZE TABLE table DEDUPLICATE BY *;
OPTIMIZE TABLE table DEDUPLICATE BY * EXCEPT colX;
OPTIMIZE TABLE table DEDUPLICATE BY * EXCEPT (colX, colY);
OPTIMIZE TABLE table DEDUPLICATE BY col1,col2,col3;
OPTIMIZE TABLE table DEDUPLICATE BY COLUMNS('column-matched-by-regex');
OPTIMIZE TABLE table DEDUPLICATE BY COLUMNS('column-matched-by-regex') EXCEPT colX;
OPTIMIZE TABLE table DEDUPLICATE BY COLUMNS('column-matched-by-regex') EXCEPT (colX, colY);
Note that * behaves just like in SELECT: MATERIALIZED, and ALIAS columns are not used for expansion.
Also, it is an error to specify empty list of columns, or write an expression that results in an empty list of columns, or deduplicate by an ALIAS column.
Column transformers other than EXCEPT are not supported.
2020-12-07 09:44:07 +03:00
Anton Popov
60b0cbb1c1
Merge pull request #15939 from Avogar/optimize_final_optimization
...
Optimize final optimization
2020-12-05 02:26:27 +03:00
Alexander Kuzmenkov
b54d50fa7f
Merge pull request #17459 from azat/countMatches-pull
...
Add countMatches function
2020-12-05 00:01:47 +03:00
Ivan
315ff4f0d9
ANTLR4 Grammar for ClickHouse and new parser ( #11298 )
2020-12-04 05:15:44 +03:00
Anton Popov
cd1917c7a6
Merge branch 'master' into optimize_final_optimization
2020-12-03 16:52:51 +03:00
Ivan
278cde366c
Update arcadia_skip_list.txt
2020-12-02 22:55:48 +03:00
alexey-milovidov
85f4045160
Merge pull request #17144 from fibersel/issue-16791
...
introduce zstd compression (for data import/export)
2020-12-02 22:41:07 +03:00
Azat Khuzhin
5365718f01
Fix optimize_distributed_group_by_sharding_key for query with OFFSET only ( #16996 )
...
* Fix optimize_distributed_group_by_sharding_key for query with OFFSET only
* Fix 01244_optimize_distributed_group_by_sharding_key flakiness
2020-12-02 20:11:39 +03:00
Azat Khuzhin
cbd4434a33
Add a test for countMatches()/countMatchesCaseInsensitive()
2020-12-01 22:25:28 +03:00
Nikita Mikhaylov
519b886641
Merge pull request #17634 from nikitamikhaylov/scipy-arcadia
...
Try to fix Yandex Synchronization check
2020-12-01 17:59:49 +03:00
alexey-milovidov
2644f446ab
Merge pull request #17453 from azat/memory-tracking-exception-locking
...
Do not allow to throw MEMORY_LIMIT_EXCEEDED if there is uncaught exception
2020-12-01 10:25:17 +03:00
nikitamikhaylov
1037a994a8
done
2020-11-30 15:53:37 +03:00
Alexander Kuzmenkov
d9dd9341ce
Fix the OpenTelemetry test
2020-11-30 12:11:17 +03:00
alesapin
25f40db2fb
Merge pull request #17499 from ClickHouse/concurrent_mutation_and_random_kill
...
Fix kill mutation on concurrent alter queries
2020-11-30 10:51:50 +03:00
alexey-milovidov
46e685e1b8
Merge pull request #17565 from ClickHouse/fix-multiline-queries-with-comments
...
Fix multiline queries with comments in interactive mode
2020-11-30 09:38:41 +03:00
alexey-milovidov
df90cbd7d3
Merge pull request #17563 from ClickHouse/parser-alter-missing-code
...
Allow query parameters in UPDATE statement.
2020-11-30 09:38:14 +03:00
Alexey Milovidov
7f1a8b15fd
Fix bad test
2020-11-29 21:21:59 +03:00
alexey-milovidov
484f7601d3
Merge pull request #17562 from ClickHouse/test-memory-accounting-zeros
...
Add a test for #11803
2020-11-29 21:17:25 +03:00
alexey-milovidov
b682539d4a
Merge pull request #17561 from ClickHouse/test-9490
...
Add a test for #9490
2020-11-29 21:16:39 +03:00
alexey-milovidov
b03aaf2b8a
Merge pull request #17557 from ClickHouse/test-chertus
...
Add a test for #12297
2020-11-29 21:16:03 +03:00
alexey-milovidov
5a1ebb5722
Merge pull request #17541 from ClickHouse/add-test-engine-join-uuid
...
Add a test for StorageJoin and UUID
2020-11-29 21:15:40 +03:00
alexey-milovidov
038a56d243
Merge pull request #16951 from CurtizJ/fix-groupby-optimization
...
Fix optimization with 'optimize_aggregators_of_group_by_keys' and joins
2020-11-29 20:47:16 +03:00
alexey-milovidov
1c7844b91e
Merge pull request #16956 from CurtizJ/fix-order-by
...
Fix order by optimization with monotonous functions
2020-11-29 20:45:47 +03:00
alexey-milovidov
ced739f231
Merge branch 'master' into test-distributed_buffer_cannot_find_column
2020-11-29 17:54:16 +03:00
Alexey Milovidov
98f9670f69
Add a test
2020-11-29 17:50:50 +03:00
alexey-milovidov
a965dba0b9
Update 01599_mutation_query_params.sh
2020-11-29 16:45:08 +03:00
Alexey Milovidov
4f47545bc8
Add a test for #10976
2020-11-29 16:42:56 +03:00
Alexey Milovidov
5edda3c6bd
Add a test for #11803
2020-11-29 16:04:57 +03:00
Alexey Milovidov
3ccc4d66a1
Add a test for #9490
2020-11-29 16:02:35 +03:00
Alexey Milovidov
3bd9a7d3bd
Add a test for #12297
2020-11-29 15:23:16 +03:00
a.palagashvili
502355922d
added errors checking, removed redundant variables
2020-11-29 15:14:42 +03:00
alexey-milovidov
00da5148a1
Merge pull request #17347 from azat/substringCount
...
Implement countSubstrings()
2020-11-29 14:32:33 +03:00
alexey-milovidov
fabceebbce
Merge pull request #17145 from amosbird/cddt
...
Fix unmatched type comparison in KeyCondition
2020-11-29 14:29:35 +03:00
alexey-milovidov
43aa7740fb
Merge pull request #17526 from ClickHouse/obfuscator-remove-some-words
...
Query obfuscator: remove some words
2020-11-29 14:25:09 +03:00
Alexey Milovidov
e635b47299
Add a test for StorageJoin and UUID
2020-11-29 10:43:34 +03:00
alexey-milovidov
19bb345440
Merge pull request #17506 from ClickHouse/fix-bad-test
...
Fix bad test 01317_no_password_in_command_line.sh
2020-11-28 15:38:22 +03:00
Azat Khuzhin
6803e563a5
Add a test for INSERT SELECT with very small memory limits
...
I found one query, using which, pretty easy to abnormal terminate the
server, for example executing the following query:
insert into placeholder_table_name select * from numbers_mt(65535) settings max_memory_usage=1, max_untracked_memory=1 ; -- { serverError 60 }
Will lead to abnormal server termination:
22:51:36.377873 [ 1853495 ] {edae87ee-05b3-4bfa-be51-3f81d25bca2e} <Debug> executeQuery: (from [::1]:48852) insert into placeholder_table_name select * from numbers_mt(65535) format Null settings max_memory_usage=1, max_untracked_memory=1
22:51:36.390607 [ 1853446 ] {} <Trace> BaseDaemon: Received signal -1
22:51:36.390644 [ 1853446 ] {} <Fatal> BaseDaemon: (version 20.12.1.5205 (official build), build id: B23CB64BD60D7F4C) (from thread 1853495) Terminate called for uncaught exception:
e.displayText() = DB::Exception: Memory limit (for query) exceeded: would use 2.01 MiB (attempt to allocate chunk of 48 bytes), maximum: 1.00 B, Stack trace (when copying this message, always include the lines below):
uild_docker/../contrib/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x107323d0 in /src/ch/tmp/upstream/clickhouse
uild_docker/../src/Common/Exception.cpp:39: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x74f5085 in /src/ch/tmp/upstream/clickhouse
uild_docker/../contrib/libcxx/include/string:2134: DB::Exception::Exception<char const*, char const*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, long&, std::__1::basic_string<char, std::__1::char_traits<char>,
22:51:36.390667 [ 1853446 ] {} <Trace> BaseDaemon: Received signal 6
22:51:36.390801 [ 1854585 ] {} <Fatal> BaseDaemon: ########################################
22:51:36.390827 [ 1854585 ] {} <Fatal> BaseDaemon: (version 20.12.1.5205 (official build), build id: B23CB64BD60D7F4C) (from thread 1853495) (query_id: edae87ee-05b3-4bfa-be51-3f81d25bca2e) Received signal Aborted (6)
22:51:36.390842 [ 1854585 ] {} <Fatal> BaseDaemon:
22:51:36.390859 [ 1854585 ] {} <Fatal> BaseDaemon: Stack trace: 0x7ffff7dde615 0x7ffff7dc7862 0x768a37d 0x11aea23c 0x11ad6df1 0x11ad72bc 0x11ad7806 0x11af6657 0x11ae99ea 0x74df512 0x74de86c 0x74dd02c 0x11acf19c 0xd692953 0xd7bf172 0xd7bc031 0xd7b8c11 0xdf37cf9 0xdf38e9e 0x1067a1cb 0x1067a660 0x107bec66 0x107ba3d0 0x7ffff7f843e9 0x7ffff7ea1293
22:51:36.390888 [ 1854585 ] {} <Fatal> BaseDaemon: 3. raise @ 0x3d615 in /usr/lib/libc-2.32.so
22:51:36.390900 [ 1854585 ] {} <Fatal> BaseDaemon: 4. __GI_abort @ 0x26862 in /usr/lib/libc-2.32.so
22:51:36.391618 [ 1854585 ] {} <Fatal> BaseDaemon: 5. /build/build_docker/../src/IO/WriteBuffer.h:46: terminate_handler() (.cold) @ 0x768a37d in /src/ch/tmp/upstream/clickhouse
22:51:36.392363 [ 1854585 ] {} <Fatal> BaseDaemon: 6. /build/build_docker/../contrib/libcxxabi/src/cxa_handlers.cpp:61: std::__terminate(void (*)()) @ 0x11aea23c in /src/ch/tmp/upstream/clickhouse
22:51:36.393103 [ 1854585 ] {} <Fatal> BaseDaemon: 7. /build/build_docker/../contrib/libcxxabi/src/cxa_personality.cpp:325: call_terminate @ 0x11ad6df1 in /src/ch/tmp/upstream/clickhouse
22:51:36.393812 [ 1854585 ] {} <Fatal> BaseDaemon: 8. /build/build_docker/../contrib/libcxxabi/src/cxa_personality.cpp:887: scan_eh_tab @ 0x11ad72bc in /src/ch/tmp/upstream/clickhouse
22:51:36.394541 [ 1854585 ] {} <Fatal> BaseDaemon: 9. /build/build_docker/../contrib/libcxxabi/src/cxa_personality.cpp:970: __gxx_personality_v0 @ 0x11ad7806 in /src/ch/tmp/upstream/clickhouse
22:51:36.395265 [ 1854585 ] {} <Fatal> BaseDaemon: 10. /build/obj-x86_64-linux-gnu/../contrib/libunwind/src/UnwindLevel1.c:101: _Unwind_RaiseException @ 0x11af6657 in /src/ch/tmp/upstream/clickhouse
22:51:36.395979 [ 1854585 ] {} <Fatal> BaseDaemon: 11. /build/build_docker/../contrib/libcxxabi/src/cxa_exception.cpp:151: __cxa_throw @ 0x11ae99ea in /src/ch/tmp/upstream/clickhouse
22:51:36.396038 [ 1854585 ] {} <Fatal> BaseDaemon: 12. /build/build_docker/../src/Common/MemoryTracker.cpp:171: MemoryTracker::alloc(long) (.cold) @ 0x74df512 in /src/ch/tmp/upstream/clickhouse
22:51:36.396212 [ 1854585 ] {} <Fatal> BaseDaemon: 13. /build/build_docker/../src/Common/MemoryTracker.cpp:177: MemoryTracker::alloc(long) @ 0x74de86c in /src/ch/tmp/upstream/clickhouse
22:51:36.396225 [ 1854585 ] {} <Fatal> BaseDaemon: 14. /build/build_docker/../base/common/../common/memory.h:20: operator new(unsigned long) @ 0x74dd02c in /src/ch/tmp/upstream/clickhouse
22:51:36.396948 [ 1854585 ] {} <Fatal> BaseDaemon: 15. /build/build_docker/../contrib/libcxx/include/string:1495: std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::basic_string(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x11acf19c in /src/ch/tmp/upstream/clickhouse
22:51:36.397683 [ 1854585 ] {} <Fatal> BaseDaemon: 16. /build/build_docker/../contrib/libcxx/include/list:361: DB::ProcessListEntry::~ProcessListEntry() @ 0xd692953 in /src/ch/tmp/upstream/clickhouse
22:51:36.399234 [ 1854585 ] {} <Fatal> BaseDaemon: 17. /build/build_docker/../contrib/libcxx/include/memory:3483: std::__1::shared_ptr<DB::ProcessListEntry>::~shared_ptr() @ 0xd7bf172 in /src/ch/tmp/upstream/clickhouse
22:51:36.400472 [ 1854585 ] {} <Fatal> BaseDaemon: 18. /build/build_docker/../src/Interpreters/executeQuery.cpp:428: DB::executeQueryImpl(char const*, char const*, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool, DB::ReadBuffer*) (.cold) @ 0xd7bc031 in /src/ch/tmp/upstream/clickhouse
22:51:36.401525 [ 1854585 ] {} <Fatal> BaseDaemon: 19. /build/build_docker/../src/Interpreters/executeQuery.cpp:814: DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) @ 0xd7b8c11 in /src/ch/tmp/upstream/clickhouse
22:51:36.402887 [ 1854585 ] {} <Fatal> BaseDaemon: 20. /build/build_docker/../src/Server/TCPHandler.cpp:254: DB::TCPHandler::runImpl() @ 0xdf37cf9 in /src/ch/tmp/upstream/clickhouse
22:51:36.404453 [ 1854585 ] {} <Fatal> BaseDaemon: 21. /build/build_docker/../src/Server/TCPHandler.cpp:1336: DB::TCPHandler::run() @ 0xdf38e9e in /src/ch/tmp/upstream/clickhouse
22:51:36.405158 [ 1854585 ] {} <Fatal> BaseDaemon: 22. /build/build_docker/../contrib/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x1067a1cb in /src/ch/tmp/upstream/clickhouse
22:51:36.405918 [ 1854585 ] {} <Fatal> BaseDaemon: 23. /build/build_docker/../contrib/libcxx/include/atomic:856: Poco::Net::TCPServerDispatcher::run() @ 0x1067a660 in /src/ch/tmp/upstream/clickhouse
22:51:36.406690 [ 1854585 ] {} <Fatal> BaseDaemon: 24. /build/build_docker/../contrib/poco/Foundation/include/Poco/Mutex_POSIX.h:59: Poco::PooledThread::run() @ 0x107bec66 in /src/ch/tmp/upstream/clickhouse
22:51:36.407413 [ 1854585 ] {} <Fatal> BaseDaemon: 25. /build/build_docker/../contrib/poco/Foundation/include/Poco/AutoPtr.h:215: Poco::ThreadImpl::runnableEntry(void*) @ 0x107ba3d0 in /src/ch/tmp/upstream/clickhouse
22:51:36.407428 [ 1854585 ] {} <Fatal> BaseDaemon: 26. start_thread @ 0x93e9 in /usr/lib/libpthread-2.32.so
22:51:36.407436 [ 1854585 ] {} <Fatal> BaseDaemon: 27. clone @ 0x100293 in /usr/lib/libc-2.32.so
But after other patches in this series it wont.
2020-11-28 11:15:26 +03:00
Alexey Milovidov
7c0138542b
Query obfuscator: remove some words
2020-11-28 10:55:15 +03:00
alexey-milovidov
c189f6405f
Merge pull request #16767 from azat/optimize_trivial_count_query-fix
...
Fix optimize_trivial_count_query with partition predicate
2020-11-28 08:43:12 +03:00
alexey-milovidov
62a45e8ff7
Merge pull request #17414 from azat/fix-INSERT-SETTINGS-parsing
...
Fix parsing of SETTINGS clause of the INSERT ... SELECT ... SETTINGS query
2020-11-28 08:38:42 +03:00
alexey-milovidov
7a6c72ce88
Merge pull request #17471 from CurtizJ/fix-order-by-functions
...
Fix 'optimize_redundant_functions_in_order_by'
2020-11-28 08:34:26 +03:00
Alexey Milovidov
44b059c681
Fix bad test 01317_no_password_in_command_line.sh
2020-11-28 08:27:59 +03:00
Nikita Mikhaylov
6f3db7ff50
Merge pull request #17330 from CurtizJ/move-to-prewhere
...
Allow to move conditions to prewhere with compact parts
2020-11-28 02:02:42 +03:00
alesapin
6567796014
Fix kill mutation on concurrent alter queries
2020-11-27 18:46:52 +03:00
Anton Popov
458395dcaf
fix test
2020-11-27 17:31:30 +03:00
Anton Popov
601d633690
fix optimize_redundant_functions_in_order_by
2020-11-27 14:29:39 +03:00
Alexander Kuzmenkov
19bdb34b19
Merge pull request #16183 from hexiaoting/dev_replace
...
Fix: throw error when column transformer use non-exist column
2020-11-27 14:09:58 +03:00
Nikolai Kochetov
85283b3944
Merge pull request #17397 from ClickHouse/fix-types-for-read-from-join
...
Fix crash while reading from JOIN table with LowCardinality types
2020-11-27 13:37:05 +03:00