thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-09-21 17:20:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
52,959 Commits 8,942 Branches 1,766 Tags 2 GiB
1c52842e83
Commit Graph

4872 Commits

Author SHA1 Message Date
alexey-milovidov
61a8406c1e
Merge pull request #17485 from ClickHouse/aku/perf-thresholds 
Adjust perf test thresholds
 2020-11-28 08:34:44 +03:00
alexey-milovidov
7a6c72ce88
Merge pull request #17471 from CurtizJ/fix-order-by-functions 
Fix 'optimize_redundant_functions_in_order_by'
 2020-11-28 08:34:26 +03:00
Alexey Milovidov
44b059c681 Fix bad test 01317_no_password_in_command_line.sh 2020-11-28 08:27:59 +03:00
Nikita Mikhaylov
6f3db7ff50
Merge pull request #17330 from CurtizJ/move-to-prewhere 
Allow to move conditions to prewhere with compact parts
 2020-11-28 02:02:42 +03:00
tavplubix
aaaa7ad59f
Merge pull request #16806 from TCeason/add_test_for_materialize_mysql 
add network partition integration test for MaterializeMySQL
 2020-11-27 20:39:06 +03:00
alesapin
6567796014 Fix kill mutation on concurrent alter queries 2020-11-27 18:46:52 +03:00
Anton Popov
458395dcaf fix test 2020-11-27 17:31:30 +03:00
Nikita Mikhaylov
0b6f5c75b9
Merge pull request #17435 from vitlibar/grpc-protocol-2 
Implement GRPC protocol (corrections)
 2020-11-27 15:19:50 +03:00
Alexander Kuzmenkov
a3277b183d Adjust perf test thresholds 2020-11-27 15:08:42 +03:00
alesapin
d199a2bf76 Merge branch 'master' into complete_zk_api 2020-11-27 14:55:55 +03:00
alesapin
17fa29f44f Less pulls of network image 2020-11-27 14:38:04 +03:00
Anton Popov
601d633690 fix optimize_redundant_functions_in_order_by 2020-11-27 14:29:39 +03:00
alesapin
f12ba9f2c1 Fix test 2020-11-27 14:26:42 +03:00
Alexander Kuzmenkov
19bdb34b19
Merge pull request #16183 from hexiaoting/dev_replace 
Fix: throw error when column transformer use non-exist column
 2020-11-27 14:09:58 +03:00
alesapin
50cf7fda39 Less flaky test_realod_zookeeper 2020-11-27 14:05:59 +03:00
Nikolai Kochetov
f3bea2210c Update tests. 2020-11-27 13:55:45 +03:00
Nikolai Kochetov
85283b3944
Merge pull request #17397 from ClickHouse/fix-types-for-read-from-join 
Fix crash while reading from JOIN table with LowCardinality types
 2020-11-27 13:37:05 +03:00
Nikolai Kochetov
60af8219ee
Merge pull request #17439 from ClickHouse/chengxianglibra-dist-agg-issue 
Fix incorrect optimization of distinct
 2020-11-27 13:26:19 +03:00
Nikolai Kochetov
c206b2506b Update tests. 2020-11-27 13:10:09 +03:00
alexey-milovidov
dfae1efbbd
Merge pull request #17070 from fastio/master 
Support multiple ZooKeeper clusters
 2020-11-27 10:38:01 +03:00
taichong
c8f7a56a25 fix CI timeout err 2020-11-27 11:34:49 +08:00
hexiaoting
2523f8f0c6 Merge remote-tracking branch 'origin/master' into dev_replace 2020-11-27 10:51:04 +08:00
alexey-milovidov
572bdb4090
Merge pull request #17423 from ClickHouse/remove-time-series-group-sum 
Remove timeSeriesGroupSum
 2020-11-26 23:07:46 +03:00
Azat Khuzhin
838596c7a4 Implement countSubstrings() 
Function to count number of substring occurrences in the string:
- in case of needle is multi char - counts non-intersecting substrings
- the code is based on position helpers.

The following new functions is available:
- countSubstrings()
- countSubstringsCaseInsensitive()
- countSubstringsCaseInsensitiveUTF8()

v0: substringCount()

v2:
- add substringCountCaseInsensitiveUTF8
- improve tests
- fix coding style issues
- fix multichar needle

v3: rename to countSubstrings (by analogy with countEqual())
 2020-11-26 22:58:16 +03:00
alexey-milovidov
1cd09fa367
Merge pull request #15127 from ucasFL/add-mutation-for-storagememory 
Add mutation support for StorageMemory
 2020-11-26 22:22:59 +03:00
Nikolai Kochetov
9291bbb04b
Merge pull request #16804 from vdimir/ip-dict-no-trie 
sorted-array based ip_dict
 2020-11-26 19:26:06 +03:00
Alexander Kuzmenkov
15a0f14445
Merge pull request #15419 from myrrc/improvement/diff-types-in-avg-weighted 
Allow different types in avgWeighted. Allow avg and avgWeighed to operate on extended integral types.
 2020-11-26 17:16:48 +03:00
alesapin
90bddce5ed
Merge pull request #17410 from ClickHouse/test_mutation_for_empty_partitions 
Add test for mutation with empty partition
 2020-11-26 16:42:43 +03:00
Nikita Mikhaylov
084c75fa6e
Merge pull request #17309 from nikitamikhaylov/merging-sequential-consistency 
Merging #16309
 2020-11-26 15:26:05 +03:00
Nikita Mikhaylov
adb82649c4
Merge pull request #16883 from nikitamikhaylov/refactor-rank-corr 
mannWhitneyUTest + studentTTest + welchTTest + small rankCorr refactor
 2020-11-26 15:17:18 +03:00
Nikolai Kochetov
83f2fc3d9c Update test. 2020-11-26 12:51:55 +03:00
Nikolai Kochetov
42399e8866 Merge branch 'dist-agg-issue' of https://github.com/chengxianglibra/ClickHouse into chengxianglibra-dist-agg-issue 2020-11-26 11:35:47 +03:00
Amos Bird
022ba2b0a9
Fix unmatched type comparison in KeyCondition 2020-11-26 16:15:50 +08:00
alexey-milovidov
ee3a0b790b
Merge pull request #17376 from azat/toUnixTimestamp-Date-fix 
Prohibit toUnixTimestamp(Date())
 2020-11-26 10:50:00 +03:00
Vitaly Baranov
1fc43b3c93 Remove trailing whitespaces in config.xml 2020-11-26 10:27:04 +03:00
Vitaly Baranov
49cf980761 Use port 9100 for grpc by default. 2020-11-26 10:14:27 +03:00
alexey-milovidov
747453b008
Merge pull request #17051 from hexiaoting/bugfix_fuzzbits 
Bug fix for funciton fuzzBits
 2020-11-26 06:41:20 +03:00
Alexey Milovidov
85d3e62275 Remove timeSeriesGroupSum 2020-11-26 05:46:28 +03:00
alexey-milovidov
fb3a69b298
Merge pull request #17254 from azat/fix-dist-query-cancelation 
Fix "Unexpected packet Data received from client"  for Distributed queries
 2020-11-26 03:46:27 +03:00
alexey-milovidov
c438b43892
Merge pull request #17375 from ClickHouse/always-working-test 
Added a test for what was always working
 2020-11-26 03:29:14 +03:00
Nikita Mikhaylov
ede023a237
rerun tests to be sure 2020-11-26 01:24:42 +03:00
Alexander Kuzmenkov
394b81ac46 Merge remote-tracking branch 'origin/master' into HEAD 2020-11-25 23:38:55 +03:00
Azat Khuzhin
0b47f4a9e9 Fix optimize_trivial_count_query with partition predicate 
Consider the following example:

    CREATE TABLE test(p DateTime, k int) ENGINE MergeTree PARTITION BY toDate(p) ORDER BY k;
    INSERT INTO test VALUES ('2020-09-01 00:01:02', 1), ('2020-09-01 20:01:03', 2), ('2020-09-02 00:01:03', 3);

- SELECT count() FROM test WHERE toDate(p) >= '2020-09-01' AND p <= '2020-09-01 00:00:00'
  In this case rpn will be (FUNCTION_IN_RANGE, FUNCTION_UNKNOWN (due to strict), FUNCTION_AND)
  and for optimize_trivial_count_query we cannot use index if there is at least one FUNCTION_UNKNOWN.
  since there is no post processing and return count() based on only the first predicate is wrong.

  Before this patch FUNCTION_UNKNOWN was allowed for optimize_trivial_count_query, and the result was wrong.

And two examples above just to show the difference, the behaviour hadn't been changed with this patch:

- SELECT * FROM test WHERE toDate(p) >= '2020-09-01' AND p <= '2020-09-01 00:00:00'
  In this case will be (FUNCTION_IN_RANGE, FUNCTION_IN_RANGE (due to non-strict), FUNCTION_AND)
  so it will prune everything out and nothing will be read.

- SELECT * FROM test WHERE toDate(p) >= '2020-09-01' AND toUnixTimestamp(p)%5==0
  In this case will be (FUNCTION_IN_RANGE, FUNCTION_UNKNOWN, FUNCTION_AND)
  and all, two, partitions will be scanned, but due to filtering later none of rows will be matched.
 2020-11-25 23:09:17 +03:00
Azat Khuzhin
32f69dc76f Cleanup 01505_trivial_count_with_partition_predicate 2020-11-25 23:09:17 +03:00
Nikolai Kochetov
729272391f
Merge branch 'master' into ip-dict-no-trie 2020-11-25 23:07:19 +03:00
Nikolai Kochetov
99f1151236
Update arcadia_skip_list.txt 2020-11-25 23:06:37 +03:00
Azat Khuzhin
3f67e320dd Fix parsing of SETTINGS clause of the INSERT ... SELECT ... SETTINGS query 
Before this patch the following query ignores the settings for INSERT:

   insert into test_parallel_insert select * from numbers_mt(65535*2) settings max_insert_threads=10

And the reason is that SETTINGS was parsed by the SELECT parser.
Fix this by push down the SETTINGS from the SELECT to INSERT.

Also note that since INSERT parser does not use ParserQueryWithOutput the
following works:

   insert into test_parallel_insert select * from numbers_mt(65535*2) format Null settings max_insert_threads=10
 2020-11-25 22:53:58 +03:00
Azat Khuzhin
688cb6b4d9 Update date_time_short perf test for toUnixTimestamp(Date()) 2020-11-25 21:17:11 +03:00
nikitamikhaylov
44f4064e9b update test 2020-11-25 20:51:07 +03:00
nikitamikhaylov
02a80230f8 remove comments 2020-11-25 20:51:07 +03:00
nikitamikhaylov
71d6126c6b better 2020-11-25 20:51:07 +03:00
nikitamikhaylov
4246971961 fix tests 2020-11-25 20:51:07 +03:00
nikitamikhaylov
de9dc3a43f update copy pasted test 2020-11-25 20:51:06 +03:00
nikitamikhaylov
f764332070 better merge 2020-11-25 20:51:06 +03:00
nikitamikhaylov
3f874af323 new interface for the function 
(cherry picked from commit 89547e77cf)
 2020-11-25 20:51:06 +03:00
nikitamikhaylov
9177ba3c02 test added 2020-11-25 20:51:06 +03:00
nikitamikhaylov
13081370ba mann-whitney 2020-11-25 20:51:06 +03:00
nikitamikhaylov
9afe579225 small refactor 2020-11-25 20:51:06 +03:00
alesapin
68c8219edf Add test for mutation with empty partition 2020-11-25 18:13:27 +03:00
alesapin
dd4c5bdeb9
Merge pull request #17249 from amosbird/insub 
Fix const column in set index.
 2020-11-25 17:05:23 +03:00
Nikolai Kochetov
8e5f7acd07 Added test. 2020-11-25 16:34:07 +03:00
alesapin
fb86eaf6fc Better errors and better config 2020-11-25 16:19:09 +03:00
Nikita Mikhaylov
27acf6462f
Merge pull request #17248 from kaka11chen/15235 
Fix #15235. When clickhouse-copier handle non-partitioned table, throws segfault error.
 2020-11-25 14:31:59 +03:00
Alexander Kuzmenkov
edce1e636e
Merge pull request #16535 from ClickHouse/aku/opentelemetry 
OpenTelemetry improvements
 2020-11-25 14:10:17 +03:00
myrrc
1a9c313529 fix test reference 2020-11-25 12:19:19 +03:00
alesapin
4d40d9be3f Merge branch 'master' into complete_zk_api 2020-11-25 10:31:16 +03:00
alesapin
3e1b2f515b
Merge pull request #17311 from ClickHouse/fix_integration_tests 
Fix some flaky tests
 2020-11-25 10:09:27 +03:00
Vitaly Baranov
08b3707842
Merge pull request #15111 from vitlibar/grpc-protocol 
Implement GRPC protocol.
 2020-11-25 09:00:32 +03:00
Azat Khuzhin
9649b96bbb Update expectations for toUnixTimestamp(Date()) in 00921_datetime64_compatibility 2020-11-25 07:42:36 +03:00
chenqi
302cd55f45 Fix #15235. When clickhouse-copier handle non-partitioned table, throw segfault error. 2020-11-25 11:52:00 +08:00
Azat Khuzhin
17f6c82ffa Prohibit toUnixTimestamp(Date()) 
Making it implicitly cast to Date() does not looks correct, since before
it returns somewhat unexpected result:

    SELECT toUnixTimestamp(today())

    ┌─toUnixTimestamp(today())─┐
    │                    18591 │
    └──────────────────────────┘
 2020-11-25 00:26:14 +03:00
Alexey Milovidov
99073c26ee Added a test for what was always working 2020-11-25 00:00:55 +03:00
Alexander Kuzmenkov
1d7dcef4a0 Merge remote-tracking branch 'origin/master' into HEAD 2020-11-24 21:45:10 +03:00
Alexander Kuzmenkov
1d6558f694 not sure what's going on 2020-11-24 21:45:01 +03:00
Anton Popov
f2106f9683 add test 2020-11-24 20:41:11 +03:00
Vitaly Baranov
65719da0d8
Merge pull request #16977 from MyroTk/rbac_attach_create_drop_detach_testflows 
RBAC Testflows tests for ATTACH, CREATE, DROP, and DETACH
 2020-11-24 18:31:44 +03:00
Vitaly Baranov
4b3488934c Disable running test test_odbc_interaction.test_bridge_dies_with_parent with asan. 2020-11-24 17:55:02 +03:00
Vitaly Baranov
3c85165e60 Redirect stderr, stdout for clickhouse-odbc-bridge to files. 2020-11-24 17:55:02 +03:00
Vitaly Baranov
950bccb130 Add test for authentication. 2020-11-24 17:55:02 +03:00
Vitaly Baranov
a0e384b0c0 Added support for external tables. 2020-11-24 17:55:02 +03:00
Vitaly Baranov
23842e7ac6 Added support for input() function. 2020-11-24 17:55:02 +03:00
Vitaly Baranov
0e3a8840b5 Support cancellation of executing query via gRPC. 2020-11-24 17:55:02 +03:00
Vitaly Baranov
88f1eeec2f Add tests for parallel executing of queries. 2020-11-24 17:55:02 +03:00
Vitaly Baranov
eab3006a4d Support SSL and compression when using gRPC protocol. 2020-11-24 17:55:02 +03:00
Vitaly Baranov
b0cb3eb306 Extend the protocol with streaming and nonstreaming functions. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
b51e14253d Added support for sessions in gRPC protocol. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
9285f7edc1 Send profile info to client. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
98e2cc4117 Fix sending progress. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
4f0405af93 Send logs via gRPC protocol too. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
218d9ea3e8 Added input_data_delimiter to protocol. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
797c84889f Use ReadBuffer to join input data from multiple query infos. 
That fixes inserting of a row splitted between two query infos.
 2020-11-24 17:55:01 +03:00
Vitaly Baranov
63c8d8124a Added test for inserting to columns with defaults. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
de4586739a Fix using output format. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
8146093794 Provide more information about errors. 2020-11-24 17:55:01 +03:00
Vitaly Baranov
a327f24e3c Refine the protocol. Code cleanup in tests. 2020-11-24 17:53:34 +03:00
Vitaly Baranov
db9f762e73 Fix tests. 2020-11-24 17:53:34 +03:00
Vitaly Baranov
13f2352c57 Move files to right places. 2020-11-24 17:53:34 +03:00
mnkonkova
6cd1557d67 Implement GRPC protocol. 2020-11-24 17:53:34 +03:00
Amos Bird
6b4512809c
Fix const column in set index. 2020-11-24 22:31:59 +08:00
Alexander Kuzmenkov
9b0368ffa5
Merge pull request #16338 from ucasFL/union-distinct 
Add UNION DISTINCT and setting `union_default_mode`
 2020-11-24 17:30:24 +03:00
myrrc
03139c2c1e Merge branch 'improvement/diff-types-in-avg-weighted' of github:myrrc/clickhouse into improvement/diff-types-in-avg-weighted 2020-11-24 17:08:38 +03:00
myrrc
420f2489a7 fixed decimal scales calc, updated the tests 2020-11-24 17:07:59 +03:00
Anton Popov
4e07681444 remove outdated test 2020-11-24 16:30:52 +03:00
myrrc
fbb0e6e6aa Merge remote-tracking branch 'upstream/master' into improvement/diff-types-in-avg-weighted 2020-11-24 16:04:17 +03:00
Alexander Tokmakov
e82c63e05c fixes 2020-11-24 15:28:54 +03:00
Alexander Tokmakov
dce0bf8efc Merge branch 'master' into allow_atomic_database_inside_materialize_mysql 2020-11-24 15:28:31 +03:00
Anton Popov
6e1a9240e1
Merge branch 'master' into fix-groupby-optimization 2020-11-24 15:24:07 +03:00
Anton Popov
ff36dee2ec move test to bugs 2020-11-24 15:22:13 +03:00
Nikita Mikhaylov
6787f98476
Merge pull request #14958 from nikitamikhaylov/refactor_cache_dictionary 
Refactor cache dictionary
 2020-11-24 14:55:02 +03:00
nikitamikhaylov
5be300ff22 Merge branch 'master' of github.com:ClickHouse/ClickHouse into merging-sequential-consistency 2020-11-24 14:46:57 +03:00
alesapin
c1a7e4f5fa Merge branch 'master' into complete_zk_api 2020-11-24 13:37:51 +03:00
alesapin
45f09fe293
Merge pull request #17220 from ianton-ru/s3_max_redirects 
S3 max redirects
 2020-11-24 12:13:10 +03:00
vdimir
52bc290616
Regenerate ya.make, add format null to ip_trie.xml 2020-11-24 11:20:11 +03:00
alesapin
b05385cec0 Remove code from test which copy-pasted from another test 2020-11-24 11:17:37 +03:00
alesapin
cb234e28ea Fix garbage test 2020-11-24 11:09:10 +03:00
alesapin
5477984856 Longer integration tests for rabbitMQ 2020-11-24 10:33:35 +03:00
taichong
b5440882cd remove kill server while insert 2020-11-24 10:11:04 +08:00
Anton Popov
2fb8717dfd
Merge pull request #17253 from amosbird/globalinfix 
global in set index.
 2020-11-24 03:06:00 +03:00
alesapin
db1aef8994 More retries 2020-11-23 23:44:46 +03:00
alesapin
3d177d414a Merge branch 'master' into fix_integration_tests 2020-11-23 23:27:46 +03:00
Anton Popov
852a08eacb allow to move conditions to prewhere with compact parts 2020-11-23 21:27:59 +03:00
Alexander Kuzmenkov
234bd09044 debug an unreproducible test failure 2020-11-23 20:48:00 +03:00
alexey-milovidov
a2f947ddd5
Merge pull request #16993 from azat/merge-distributed-join-fix 
Fix Merge(Distributed()) with JOIN
 2020-11-23 20:40:21 +03:00
alexey-milovidov
ac710ff0b7
Merge pull request #17299 from Jokser/flaky-s3-test-custom-auth-headers 
Fixed flaky test_storage_s3::test_custom_auth_headers
 2020-11-23 20:33:48 +03:00
Alexander Kuzmenkov
412c2fe079 Merge remote-tracking branch 'origin/master' into HEAD 2020-11-23 19:51:18 +03:00
alesapin
1c5da1c178 Better timeouts 2020-11-23 19:39:57 +03:00
alesapin
eb9b3e6737 Less global queries in functional tests 2020-11-23 17:57:46 +03:00
alesapin
c9b382b4af Fix flaky integration tests 2020-11-23 17:40:32 +03:00
nikitamikhaylov
065cd00257 better 2020-11-23 17:24:32 +03:00
Amos Bird
172b7e9ed1
global in set index. 2020-11-23 22:05:08 +08:00
feng lv
005f345139 Merge branch 'master' of github.com:ClickHouse/ClickHouse into add-mutation-for-storagememory 2020-11-23 13:56:19 +00:00
nikitamikhaylov
68bef22fda Merge branch 'master' of github.com:ClickHouse/ClickHouse into merging-sequential-consistency 2020-11-23 16:28:35 +03:00
tavplubix
7af89cba8a
Merge pull request #16715 from zhang2014/fix/ISSUES-16605 
ISSUES-16605 fix without returning affected rows when insert select query in MySQL handler
 2020-11-23 15:54:43 +03:00
Anton Ivashkin
aa208a9d9f Use only 's3_max_redirect' in params instead of all settings 2020-11-23 14:02:17 +03:00
Pavel Kovalenko
5f98d38689 Fixed flaky test_storage_s3::test_custom_auth_headers 2020-11-23 13:19:43 +03:00
taichong
a6c2a90756 modify integration test materialize mysql database 
1. fix asan test err

2. reset pymysql conn if ping err

3. finish mysql & ck killed test

4. modify check_query
 2020-11-23 17:55:07 +08:00
alexey-milovidov
f38db2d421
Update 01560_merge_distributed_join.sql 2020-11-23 12:51:59 +03:00
Alexey Milovidov
9969a73d50 Merge branch 'master' into azat-merge-distributed-join-fix 2020-11-23 12:43:15 +03:00
Alexey Milovidov
ed679d8edb Better test 2020-11-23 12:38:03 +03:00
Alexey Milovidov
24710046c3 Arcadia does not support distributed tests 2020-11-23 12:34:23 +03:00
Alexey Milovidov
b10283f8d5 Add a test for #13990 2020-11-23 12:32:57 +03:00
lichengxiang
fdde2d1e8a fix #17294: Distinct on subquery with group by may return duplicate result 2020-11-23 17:06:59 +08:00
alesapin
b8fb44a7b9 Merge with master 2020-11-23 11:05:50 +03:00
taichong
f99d6ea799 add ck & mysql killed while insert 2020-11-23 15:47:44 +08:00
taichong
122740767f add integration test err 2020-11-23 15:47:44 +08:00
taichong
714767f491 make it normal 2020-11-23 15:47:44 +08:00
taichong
5a50535abc add mysql kill sync id test 2020-11-23 15:47:44 +08:00
taichong
81bdf50085 add some except error test 2020-11-23 15:47:44 +08:00
taichong
e691b4a0d4 add network partition integration test for MaterializeMySQL 2020-11-23 15:47:44 +08:00
taichong
e8628f61e0 add mysql kill sync id test 2020-11-23 15:47:44 +08:00
taichong
3287f691ee add some except error test 2020-11-23 15:47:44 +08:00
taichong
5c3d4b2245 add network partition integration test for MaterializeMySQL 2020-11-23 15:47:44 +08:00
feng lv
18491d8920 fix 2020-11-23 05:30:36 +00:00
Alexey Milovidov
fbbf439338 Merge branch 'master' into Enmk-DateTime64_fixes_comparison 2020-11-23 05:46:17 +03:00
alexey-milovidov
9ed0c3c85a
Merge pull request #17238 from azat/no-log-rotate-in-tests 
Disable rotation of the logs in tests (on CI)
 2020-11-23 05:40:58 +03:00
alesapin
9c8b0da382
Merge pull request #16033 from nvartolomei/nv/parts-uuid 
Add unique identifiers IMergeTreeDataPart structure
 2020-11-22 16:13:19 +03:00
zhang2014
e1e5eede86 ISSUES-16605 try fix integration failure 2020-11-22 20:13:56 +08:00
zhang2014
5b3154b298 ISSUES-16605 try fix integration test failure 2020-11-22 20:13:56 +08:00
zhang2014
a6dbba5932 ISSUES-16605 add integration test 2020-11-22 20:13:56 +08:00
alexey-milovidov
870a56b962
Merge pull request #17255 from azat/perf-tests-settings-cleanup 
Cleanup settings handling in perf tests
 2020-11-21 22:59:44 +03:00
alexey-milovidov
1663b13215
Merge pull request #17257 from ucasFL/fix 
fix toUInt256 stack overflow
 2020-11-21 22:54:06 +03:00
fenglv
268b15f221 fix toUInt256 stack overflow 2020-11-21 12:59:40 +00:00
Azat Khuzhin
8931d3eb6f Do not use SET via <full_query> in perf tests 
Since if the connection will be closed (by some reason), then the
setting will not be applied after transparent reconnect (since only
native clickhouse-client can do this, since it parses the query, but
perf tests uses python driver).

Just use inplace SETTINGS clause or <settings>.
 2020-11-21 14:02:21 +03:00
Azat Khuzhin
ee4267c1ba Rename 01247_optimize_distributed_group_by_sharding_key to distingiush it 
Otherwise flaky tests will run then both, not a problem, but this is
just to trigger flaky check, since there is some tricky issue [1]:

    2020-11-20 00:35:51 01247_optimize_distributed_group_by_sharding_key:                       [ FAIL ] 0.67 sec. - return code 101
    2020-11-20 00:35:51 Received exception from server (version 20.12.1):
    2020-11-20 00:35:51 Code: 101. DB::Exception: Received from localhost:9000. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: Unexpected packet Data received from client.

  [1]: https://clickhouse-test-reports.s3.yandex.net/16996/8d71564056925df1415880f382aaa169cbdf37b0/functional_stateless_tests_flaky_check_(address)/test_run.txt.out.log
 2020-11-21 13:32:17 +03:00
Azat Khuzhin
a3116d5614 Tune aggregating_merge_tree_simple_aggregate_function_string to make it faster 2020-11-21 12:08:59 +03:00
Nikolai Kochetov
bac1def5f9
Merge pull request #17134 from abyss7/tcp-port 
Implement tcpPort() function for tests
 2020-11-20 20:32:55 +03:00
Pavel Kruglov
ca3fe49a2a Make setting global 2020-11-20 17:29:13 +03:00
Nicolae Vartolomei
7c8bc1c04e Use JSON metadata in WAL 2020-11-20 13:49:17 +00:00
Nicolae Vartolomei
94293ca3ce Assign UUIDs to parts only when configured to do so 
Avoid breaking backwards compatibility by default for now.
 2020-11-20 13:49:17 +00:00
Pavel Kruglov
e5588b94aa Fix style and tests 2020-11-20 13:38:53 +03:00
tavplubix
3e335c1e95
Merge pull request #17091 from zhang2014/fix/binlog_checksum 
Fix MaterializeMySQL SYNC failure when modify binlog_checksum
 2020-11-20 13:25:14 +03:00
Alexander Kuzmenkov
b05ac888bd Merge remote-tracking branch 'origin/master' into HEAD 2020-11-20 12:50:08 +03:00
Alexander Kuzmenkov
bde805cb3e
Merge pull request #17175 from ClickHouse/aku/named-tuple 
Allow formatting named tuples as JSON objects
 2020-11-20 12:48:24 +03:00
Nikolai Kochetov
31886fdca6
Merge pull request #16811 from amosbird/pf2 
Fix verbatim partition pruner.
 2020-11-20 12:07:33 +03:00
alesapin
9920d77a93 Merge branch 'master' into complete_zk_api 2020-11-20 11:48:12 +03:00
Anton Ivashkin
a7d4f4be65 Add 's3_max_redirects' test 2020-11-20 11:18:44 +03:00
alesapin
bd5d2d89ad
Merge pull request #17201 from ClickHouse/fix_read_in_order_test 
Trying to make read_in_order_many_parts more stable
 2020-11-20 10:10:17 +03:00
Pavel Kruglov
608722b6ab Fix integration tests 2020-11-20 02:59:58 +03:00
Kruglov Pavel
51bcd286f2
Merge pull request #16845 from azat/cutToFirstSignificantSubdomainWithWWW 
Add cutToFirstSignificantSubdomainWithWWW()
 2020-11-20 01:44:57 +03:00
Pavel Kruglov
3ee4972586 Merge branch 'master' of github.com:ClickHouse/ClickHouse into optimize-data-on-insert 2020-11-20 01:23:51 +03:00
Azat Khuzhin
35231662b3 Improve performance of AggregatingMergeTree w/ SimpleAggregateFunction(String) 
While reading from AggregatingMergeTree with
SimpleAggregateFunction(String) in primary key and
optimize_aggregation_in_order perf top shows:

    Samples: 1M of event 'cycles', 4000 Hz, Event count (approx.): 287759760270 lost: 0/0 drop: 0/0
      Children      Self  Shared Object         Symbol
    +   12.64%    11.39%  clickhouse            [.] memcpy
    +    9.08%     0.23%  [unknown]             [.] 0000000000000000
    +    8.45%     8.40%  clickhouse            [.] ProfileEvents::increment    # <-- this, and in debug it has not 0.08x overhead, but 5.8x overhead
    +    7.68%     7.67%  clickhouse            [.] LZ4_compress_fast_extState
    +    5.29%     5.22%  clickhouse            [.] DB::IAggregateFunctionHelper<DB::AggregateFunctionNullUnary<true, true> >::addFree

The reason is obvious, ProfileEvents is atomic counters (and also they
are nested):

<details>

```
    Samples: 7M of event 'cycles', 4000 Hz, Event count (approx.): 450726149337
    ProfileEvents::increment  /usr/bin/clickhouse [Percent: local period]
    Percent│
           │
           │
           │    Disassembly of section .text:
           │
           │    00000000078d8900 <ProfileEvents::increment(unsigned long, unsigned long)@@Base>:
           │    ProfileEvents::increment(unsigned long, unsigned long):
      0.17 │      push  %rbp
      0.00 │      mov   %rsi,%rbp
      0.04 │      push  %rbx
      0.20 │      mov   %rdi,%rbx
      0.17 │      sub   $0x8,%rsp
      0.26 │    → callq DB::CurrentThread::getProfileEvents
           │    ProfileEvents::Counters::increment(unsigned long, unsigned long):
      0.00 │      lea   0x0(,%rbx,8),%rdi
      0.05 │      nop
           │    unsigned long std::__1::__cxx_atomic_fetch_add<unsigned long, unsigned long>(std::__1::__cxx_atomic_base_impl<unsigned long>*, unsigned long, std::__1::memory_order):
      1.02 │      mov   (%rax),%rdx
     97.04 │      lock  add   %rbp,(%rdx,%rdi,1)
           │    ProfileEvents::Counters::increment(unsigned long, unsigned long):
      0.21 │      mov   0x10(%rax),%rax
      0.04 │      test  %rax,%rax
      0.00 │    → jne   78d8920 <ProfileEvents::increment(unsigned long, unsigned long)@@Base+0x20>
           │    ProfileEvents::increment(unsigned long, unsigned long):
      0.38 │      add   $0x8,%rsp
      0.00 │      pop   %rbx
      0.04 │      pop   %rbp
      0.38 │    ← retq
```

</details>

These ProfileEvents was ArenaAllocChunks (it shows ~1.5M events per
second), and the reason is that the table has
SimpleAggregateFunction(String) in PK, which requires Arena.
But most of the time there Arena wasn't even used, so avoid this cost by
re-creating Arena only if it was "used" (i.e. has new chunks).

Another possibility is to avoid populating Arena::head in ctor, but this
will make the Arena code more complex, so for now this was preferred.

Also as a long-term solution it worth looking at implementing them via
RCU (to move the extra overhead out from the write code path into read
side).
 2020-11-19 23:06:12 +03:00
Alexander Kuzmenkov
87a306ed23 Enable OpenTelemetry tracing in functional tests 2020-11-19 18:58:33 +03:00
vdimir
36544a45b7
Merge remote-tracking branch 'upstream/master' into ip-dict-no-trie 2020-11-19 18:56:24 +03:00
alesapin
2623d35f68
Merge pull request #17120 from ClickHouse/fix_granularity_on_block_borders 
Fix index granularity calculation on block borders
 2020-11-19 18:36:07 +03:00
Nicolae Vartolomei
425dc4b11b Add unique identifiers IMergeTreeDataPart structure 
For now uuids are not generated at all, they are present only if the
part is updated manually (as you can see in the integration test).

The only place where they can be seen today by an end user is in
`system.parts` table. I was looking for hiding this column behind an
option but couldn't find an easy way to do that.

Likely this is also required for WAL, but need to think how not to break
compatibility.

Relates to #13574, https://github.com/ClickHouse/ClickHouse/issues/13574

Next 1: In the upcoming PR the plan is to integrate de-duplication based on
these fingerprints in the query pipeline.

Next 2: We'll enable automatic generation of uuids and come up with a
way for conditionally sending uuids when processing distributed queries
only when part movement is in progress.
 2020-11-19 13:14:25 +00:00
Alexander Kuzmenkov
f326536ef0 fixup 2020-11-19 15:30:44 +03:00
Alexander Kuzmenkov
bcf15170ed Merge remote-tracking branch 'origin/master' into HEAD 2020-11-19 15:29:51 +03:00
Alexander Kuzmenkov
24293ccb30 Merge remote-tracking branch 'origin/master' into HEAD 2020-11-19 15:28:37 +03:00
Alexander Kuzmenkov
f2b4c15ab0
Merge pull request #17152 from ClickHouse/aku/json-array 
Write rows as JSON array in JSONEachRow output format
 2020-11-19 15:27:31 +03:00
alesapin
1aba61be7f Merge branch 'master' into complete_zk_api 2020-11-19 13:32:15 +03:00
alesapin
cdceafdd89 Trying to make read_in_order_many_parts more stable 2020-11-19 13:25:39 +03:00
tavplubix
8b6fbea199
trigger CI 2020-11-19 12:47:24 +03:00
Nikolai Kochetov
8a9792cd30
Merge branch 'master' into pf2 2020-11-19 12:01:15 +03:00
Peng Jian
3c86c8b3c9 fix test cases 2020-11-19 15:44:47 +08:00
Peng Jian
d8ae52118b add test 2020-11-19 15:44:47 +08:00
alesapin
978e577b0f
Merge pull request #16424 from filimonov/execute_merges_on_single_replica_v2 
execute_merges_on_single_replica
 2020-11-19 10:15:52 +03:00
zhang2014
9df9691ccb Try fix integration test 2020-11-19 13:01:23 +08:00
alexey-milovidov
c7e88087cd
Merge pull request #16636 from hombit/cmath 
Add some of missed cmath functions
 2020-11-19 07:37:27 +03:00
alexey-milovidov
29bfb5a518
Merge pull request #16997 from vdimir/source-clickhouse-usability 
source(clickhouse(...)) usability
 2020-11-19 07:37:00 +03:00
alexey-milovidov
5aff81b8b8
Merge pull request #17135 from amosbird/ccc 
Fix ColumnConst comparison.
 2020-11-19 07:36:41 +03:00
Azat Khuzhin
dd0fccae47 Disable rotation of the logs in tests (on CI) 
Right now due to rotation the archive with the clickhouse-server.log
maybe not full, for example:
- not full - https://clickhouse-test-reports.s3.yandex.net/16947/caf5f98db2ae39dd911f7d1fc0a7c2cc382b1c53/functional_stateless_tests_(ubsan)/clickhouse-server.log (no messages about server start)
- full     - https://clickhouse-test-reports.s3.yandex.net/16993/d1f52dc72d417580c4088cf3880593176416bea2/functional_stateless_tests_(thread).html

And sometimes rotated part may include relative part of the log, so it
is better to disable it.
 2020-11-19 01:08:47 +03:00
Azat Khuzhin
4a77a42f34 Test for Merge(Distributed()) with JOIN 2020-11-18 23:53:31 +03:00
alesapin
d11c547ec2 Merge branch 'master' into fix_granularity_on_block_borders 2020-11-18 23:36:45 +03:00
Azat Khuzhin
48645eae33 Add cutToFirstSignificantSubdomainWithWWW() 
Sometimes it is odd to get TLD itself from the
cutToFirstSignificantSubdomain() (since you will not get TLD itself if
you pass it directly):
- cutToFirstSignificantSubdomain('org')            -> ""
- cutToFirstSignificantSubdomain('www.org')        -> org
- cutToFirstSignificantSubdomain('kernel.org')     -> kernel.org
- cutToFirstSignificantSubdomain('www.kernel.org') -> kernel.org

So add one more function to get www.org in this case:
- cutToFirstSignificantSubdomainWithWWW('org')            -> ""
- cutToFirstSignificantSubdomainWithWWW('www.org')        -> www.org
- cutToFirstSignificantSubdomainWithWWW('kernel.org')     -> kernel.org
- cutToFirstSignificantSubdomainWithWWW('www.kernel.org') -> kernel.org

P.S. not sure about the naming though, so it will great if someone has
suggestion for the name.
 2020-11-18 21:09:27 +03:00
Alexander Kuzmenkov
1570320e20 fixes for context hierarchy 2020-11-18 20:43:18 +03:00
alesapin
2cd66f23a3 Merge branch 'master' into complete_zk_api 2020-11-18 17:50:50 +03:00
nikitamikhaylov
13e711b27e rename back config 2020-11-18 16:38:14 +03:00
Ivan Lezhankin
fdd1c182ab Merge remote-tracking branch 'upstream/master' into tcp-port 2020-11-18 16:22:00 +03:00
alesapin
7080f424e2 Fix bug for skip indices and make code more complex 2020-11-18 15:04:13 +03:00
Pavel Kruglov
cc1b16ea81 Fix test 2020-11-18 13:40:58 +03:00
Alexander Kuzmenkov
f2b3f5f8b6 Allow formatting named tuples as JSON objects 2020-11-18 13:38:30 +03:00
Mike
1aff334ee5
Delete clickhouse-test-hs.hs 2020-11-18 12:54:54 +03:00
myrrc
dbc5284b73 replaced Memory by MergeTree in the test to get perftests 2020-11-18 12:51:02 +03:00
Alexander Kuzmenkov
8ae67e119f Merge remote-tracking branch 'origin/master' into HEAD 2020-11-18 12:38:50 +03:00
Pavel Kruglov
547ec19fb3 Merge branch 'master' of github.com:ClickHouse/ClickHouse into optimize-data-on-insert 2020-11-18 12:01:59 +03:00
Mikhail Filimonov
97fef77ed1
execute_merges_on_single_replica 2020-11-18 08:45:33 +01:00
alesapin
0bc60e2d53
Merge pull request #17089 from ClickHouse/fix_ddl_worker_non_leader 
Fix ON CLUSTER queries hung for non leader replicas
 2020-11-18 10:21:35 +03:00
Alexey Milovidov
26d7795221 Merge branch 'master' into amosbird-ccc 2020-11-18 07:42:45 +03:00
Vitaliy Zakaznikov
dbb5f6722e Moving to using TestFlows version 1.6.65 
Fixing unstable test in tests/testflows/ldap/external_user_directory/tests/authentications.py
 2020-11-17 20:22:43 -05:00
Alexander Kuzmenkov
b16c5a1748 Merge remote-tracking branch 'origin/master' into HEAD 2020-11-17 22:54:54 +03:00
Alexander Kuzmenkov
51f9d54d10 test 2020-11-17 22:51:45 +03:00
Alexander Kuzmenkov
a8f29f7da6 make it a setting and not a format 2020-11-17 22:45:17 +03:00
Pavel Kruglov
4ed7657008 fix typo 2020-11-17 22:07:19 +03:00
Alexander Kuzmenkov
2bd6584636 JSONArray fixes 2020-11-17 21:33:54 +03:00
nikitamikhaylov
70a08608f8 add reload dictionary 2020-11-17 20:58:31 +03:00
nikitamikhaylov
663730c758 try to fix flaky test 2020-11-17 20:33:07 +03:00
Pavel Kruglov
53b8ff42c2 Add test and set setting to 0 in some other tests 2020-11-17 19:56:36 +03:00
Konstantin Malanchev
07f65b852c Fix tests 2020-11-17 17:48:16 +01:00
a.palagashvili
6f5390cc70 introduce zstd compression 2020-11-17 19:02:10 +03:00
Nikita Mikhaylov
68b6f08e36
Merge pull request #16721 from nikitamikhaylov/parsing-constraints 
Turn off parallel parsing when memory limit is small.
 2020-11-17 17:37:14 +03:00
nikitamikhaylov
9c6b896928 style + flaky test fix 2020-11-17 17:36:04 +03:00
Amos Bird
f24cf46db5
Fix ColumnConst comparison. 2020-11-17 22:14:31 +08:00
nikitamikhaylov
31b1f644c1 better 2020-11-17 16:59:13 +03:00
nikitamikhaylov
fbd0d14dd6 save 2020-11-17 16:59:12 +03:00
nikitamikhaylov
9b46b53fa3 better defaults 2020-11-17 16:59:12 +03:00
nikitamikhaylov
028fc00166 fix tests 2020-11-17 16:59:12 +03:00
nikitamikhaylov
c7ebcf9c7c arcadia skip 2020-11-17 16:59:12 +03:00
nikitamikhaylov
d5408b6295 unique db to test 2020-11-17 16:59:12 +03:00
nikitamikhaylov
82c6467a5d better 2020-11-17 16:59:12 +03:00
nikitamikhaylov
e75f885ac1 add test 2020-11-17 16:59:12 +03:00
nikitamikhaylov
0d1a8ea2a1 first try 2020-11-17 16:59:12 +03:00
Ivan Lezhankin
1787cd89a7 Implement tcpPort() function literal 2020-11-17 16:24:13 +03:00
tavplubix
b251478d98
Merge pull request #17072 from ClickHouse/fix_crash_16944 
Fix crash in CREATE TABLE AS
 2020-11-17 12:51:37 +03:00
alesapin
389b883534 Fix index granularity calculation on block borders 2020-11-17 12:37:13 +03:00
Nikolai Kochetov
86395e11a2
Merge pull request #17006 from azat/fix-distributed-queries-finish 
Fix Distributed queries finishing (avoid connection resets)
 2020-11-17 11:11:27 +03:00
zhang2014
b4fc2ecc00 Try fix integration test 2020-11-17 09:41:35 +08:00
vdimir
428f42053b
Merge remote-tracking branch 'upstream/master' into source-clickhouse-usability 2020-11-16 22:34:28 +03:00
alesapin
5dd32819f5 Merge branch 'master' into fix_ddl_worker_non_leader 2020-11-16 22:34:16 +03:00
vdimir
6dcb38db3f
Minor changes in IP dictionary 2020-11-16 21:08:31 +03:00
Alexander Kuzmenkov
216098afc8 Merge remote-tracking branch 'origin/master' into HEAD 2020-11-16 19:10:10 +03:00