thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-09-20 08:40:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
93,165 Commits 8,927 Branches 1,765 Tags 2 GiB
966a59e617
Commit Graph

19199 Commits

Author SHA1 Message Date
Vladimir C
3c7fdb604d
Merge pull request #35363 from vdimir/kv-join 2022-07-06 15:57:30 +02:00
mergify[bot]
e5535f5ab4
Merge branch 'master' into mapupdate_dev 2022-07-06 13:54:36 +00:00
Mikhail f. Shiryaev
0e02271b8f
Merge pull request #38898 from ClickHouse/retry-docker-server 
Retry docker buildx commands with progressive sleep in between
 2022-07-06 15:39:44 +02:00
Kruglov Pavel
d276d9ef8b
Merge pull request #38831 from Avogar/fix-map-combinator 
Add test for segfault in Map combinator
 2022-07-06 14:26:24 +02:00
Mikhail f. Shiryaev
4410caf31d
Retry docker buildx commands with progressive sleep in between 2022-07-06 13:58:46 +02:00
mergify[bot]
e746760e41
Merge branch 'master' into improve-bc-check 2022-07-06 11:53:43 +00:00
Raúl Marín
ffac598029 Fix settings profile with seconds unit 2022-07-06 12:56:54 +02:00
Andrey Zvonov
7de39d9b15 Merge branch 'master' of github.com:ClickHouse/ClickHouse into zvonand-nnd 2022-07-06 10:59:35 +03:00
Yakov Olkhovskiy
d6269c04bb style fix 2022-07-06 00:40:51 -04:00
Yakov Olkhovskiy
69d1495672 style fix 2022-07-06 00:22:50 -04:00
HeenaBansal2009
d89ba2e5d9 Review Comments 2022-07-05 21:18:39 -07:00
Yakov Olkhovskiy
19365597eb move Simple Check to python 2022-07-05 23:37:01 -04:00
Kseniia Sumarokova
a26643a7a6
Merge pull request #38363 from kssenii/pg-add-auto-close 
Support `auto_close` option for postgres engine connection
 2022-07-06 01:10:05 +02:00
mergify[bot]
5e0bfae76b
Merge branch 'master' into multi-match-with-non_const-patterns 2022-07-05 19:23:09 +00:00
Azat Khuzhin
98cd92017e Fix waiting of shared lock after exclusive lock failure 
When WRITE lock attemp fails (exclusive lock for ALTER/DELETE), and
there are multiple READ locks (shared lock for SELECT/INSERT), i.e. one
INSERT is in progress and one SELECT is queued after ALTER/DELETE
started but before it fails, this SELECT will wait until INSERT will
finishes.

This happens because in case of WRITE lock failure it does not notify
the next READ lock that can be acquired.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-05 19:12:47 +03:00
Kseniia Sumarokova
3e803006a2
Merge pull request #37171 from tchepavel/nats-integration 
Nats Integration
 2022-07-05 17:47:02 +02:00
mergify[bot]
ba7d48c2d7
Merge branch 'master' into improve-bc-check 2022-07-05 13:40:35 +00:00
Yakov Olkhovskiy
356786f0c6
Merge pull request #38837 from bharatnc/ncb/fix-docs 
update integration tests doc
 2022-07-05 09:27:09 -04:00
Vitaly Baranov
43d35eec1b Write unfinished mutations to backup. 2022-07-05 14:51:09 +02:00
vdimir
f701b58dd0
Update check_name in performance comparsion upload 2022-07-05 14:24:58 +02:00
vdimir
3731bd44c1
Revert "[testing] run only one perftest" 
This reverts commit 3b079827b78e04ac20043346998ce9d546969746.
 2022-07-05 14:00:26 +02:00
vdimir
45b2b1ca8a
[wip] persist pertest results 2022-07-05 14:00:24 +02:00
vdimir
c2d774d598
[testing] run only one perftest 2022-07-05 14:00:24 +02:00
vdimir
aac2f8aefc
Rename envs for performance comparsion 2022-07-05 14:00:24 +02:00
vdimir
b440ff83d8
fix performance_comparison_check.py 2022-07-05 14:00:24 +02:00
vdimir
0232c9267d
Pass CHPC_DATABASE_URL to perf tests 2022-07-05 14:00:15 +02:00
Robert Schulze
2b31d68cef
Fix 00928_multi_match_constant_constant.sql 
Difficult to test because x86 and ARM behave differently due to
vectorscan not available everyehwere ... The same stuff
(multiFuzzyMatch) is tested elsewhere.
 2022-07-05 13:47:48 +02:00
Azat Khuzhin
d1d717d4e8 tests: make aggregate_state_exception_memory_leak tests more deterministic 
Sometimes if the 30 seconds is not enough for the query to fail (i.e.
under TSan [1]), the error will be ignored and the test will fail.

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/38748/d863f6ce1ece810c66ddf8bd89575825d3f2595f/stateless_tests__thread__actions__%5B2/3%5D.html

Follow-up for: #11496 (cc @alexey-milovidov)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-05 13:43:47 +03:00
Mikhail f. Shiryaev
44406060af
Merge pull request #38836 from ClickHouse/get-files-changed-in-master 
Get files changed in master since release is branched
 2022-07-05 12:18:15 +02:00
Mikhail f. Shiryaev
f048db23e2
Release branches run CI on pushes, fix PRInfo 2022-07-05 12:10:33 +02:00
kssenii
e56ffd665f Fix test 2022-07-05 12:01:19 +02:00
Kruglov Pavel
6e0f27be23
Fix 2022-07-05 11:23:35 +02:00
Kseniia Sumarokova
b3fd8e3bd6
Merge pull request #38802 from kssenii/fix-s3-seekable-with-parallel 
Fix s3 seekable reads with parallel read buffer
 2022-07-05 11:23:16 +02:00
Kruglov Pavel
a6a9824261
Merge pull request #38362 from Avogar/better-exception-messages 
Better exception messages on wrong table engines/functions argument types
 2022-07-05 11:21:44 +02:00
Kruglov Pavel
c9d1cbbc49
Update 02324_map_combinator_bug.sql 2022-07-05 11:17:41 +02:00
lokax
83613753e2 test(Function): isNullable 2022-07-05 15:24:24 +08:00
Bharat Nallan Chakravarthy
51923e4399 update integration tests doc 2022-07-05 09:27:08 +05:30
Alexey Milovidov
c5cf61bde5
Merge pull request #38813 from kssenii/client-i 
clickhouse client/local interactive: support \i file
 2022-07-05 02:28:29 +03:00
mergify[bot]
f14b62b2d6
Merge branch 'master' into index-fix-1 2022-07-04 22:50:36 +00:00
Mikhail f. Shiryaev
04948e81cc
Get files changed in master since release is branched 2022-07-05 00:29:28 +02:00
Kruglov Pavel
315e6dcdb9
Fix style 2022-07-04 22:06:04 +02:00
Kruglov Pavel
4a557201c8
Fix style 2022-07-04 21:59:46 +02:00
avogar
03f82f73fb Add test for segfault in Map combinator 2022-07-04 19:56:52 +00:00
Kseniia Sumarokova
6c3d648f49
Merge branch 'master' into nats-integration 2022-07-04 21:06:07 +02:00
Kseniia Sumarokova
b8ab20bcb3
Update test.py 2022-07-04 20:56:34 +02:00
Kseniia Sumarokova
0cd228cab3
Merge pull request #38365 from kssenii/better-rabbit-adddress-setting 
Fix RabbitMQ configuration via connection string setting
 2022-07-04 20:53:43 +02:00
Kruglov Pavel
74bd625cd5
Fix test 2022-07-04 20:48:35 +02:00
vdimir
1cad4a7e9b
Mark 02241_join_rocksdb_bs long 2022-07-04 17:28:15 +00:00
vdimir
54cf5c51de
upd 02241_join_rocksdb_bs.sql 2022-07-04 17:28:15 +00:00
vdimir
65ac97ce89
direct join: handle nullable, add tests 2022-07-04 17:28:14 +00:00
vdimir
a1ef707a8d
update 02241_join_rocksdb 2022-07-04 17:28:14 +00:00
vdimir
b3c581a1f9
drop table in 02241_join_rocksdb 2022-07-04 17:28:12 +00:00
vdimir
ad06dd3fe9
add 02241_join_rocksdb 2022-07-04 17:28:11 +00:00
avogar
3eb97313d8 Allow null modifier in columns declaration for table functions 2022-07-04 14:46:10 +00:00
Dmitry Novik
c66e2c8095
Merge pull request #37155 from amosbird/projection-fix-two 
Fix intermediate header in aggregate projection
 2022-07-04 16:32:34 +02:00
Kseniia Sumarokova
35e616514c
Update 02352_interactive_queries_from_file.expect 2022-07-04 16:26:35 +02:00
Kseniia Sumarokova
707142e939
Update test.py 2022-07-04 16:08:10 +02:00
kssenii
f5bf744028 Fix 2022-07-04 16:05:08 +02:00
Robert Schulze
1eed72b525
Make more multi-search methods work with non-const needles 
After making function multi[Fuzzy]Match(Any|AnyIndex|AllIndices)() work
with non-const needles, 12 more functions started to fail in test
"00233_position_function_family":

multiSearchAny()
multiSearchAnyCaseInsensitive()
multiSearchAnyUTF8
multiSearchAnyCaseInsensitiveUTF8()

multiSearchFirstPosition()
multiSearchFirstPositionCaseInsensitive()
multiSearchFirstPositionUTF8()
multiSearchFirstPositionCaseInsensitiveUTF8()

multiSearchFirstIndex()
multiSearchFirstIndexCaseInsensitive()
multiSearchFirstIndexUTF8()
multiSearchFirstIndexCaseInsensitiveUTF8()

Failing queries take the form
  select 0 = multiSearchAny('\0', CAST([], 'Array(String)'));
 2022-07-04 14:00:21 +00:00
kssenii
2746ede854 Better test 2022-07-04 15:59:52 +02:00
kssenii
ac8a794780 Client -i file 2022-07-04 15:40:23 +02:00
Kruglov Pavel
6c4b6fa834
Merge branch 'master' into better-exception-messages 2022-07-04 15:07:31 +02:00
Kruglov Pavel
4080f055b6
Merge pull request #38477 from Avogar/sql-insert-format 
Add SQLInsert output format
 2022-07-04 15:06:33 +02:00
avogar
fb2f60c613 Remove old script 2022-07-04 13:03:55 +00:00
avogar
17da24b584 Try to improve backward compatibility check 2022-07-04 13:02:22 +00:00
kssenii
2c4c907e10 Merge 2022-07-04 14:15:31 +02:00
kssenii
cfff7c4c28 Merge master 2022-07-04 14:13:26 +02:00
kssenii
6b37ce2f8e Drop tables in test 2022-07-04 14:06:20 +02:00
Alexander Tokmakov
a7604a19e9
Update 01710_projection_fetch_long.sql 2022-07-04 14:23:30 +03:00
Ilya Yatsishin
703c3f1938
Merge pull request #38582 from qoega/integration-postgres-better 2022-07-04 13:00:54 +02:00
Ilya Yatsishin
0e4c65a378
Update tests/integration/test_postgresql_database_engine/test.py 
Co-authored-by: alesapin <alesapin@gmail.com>
 2022-07-04 12:57:55 +02:00
Robert Schulze
d547aa7849
Allow non-const pattern array argument in multi[Fuzzy]Match*() 
Resolves #38046
 2022-07-04 10:43:16 +00:00
Mikhail f. Shiryaev
1018eff5b3
Merge pull request #38273 from ClickHouse/38218-oo-space 
Improve runners AMI and init scripts
 2022-07-04 12:39:58 +02:00
Mikhail f. Shiryaev
7c812df47e
Merge pull request #38580 from ClickHouse/artifactory-arm64 
Push arm64 packages to artifactory and release assets
 2022-07-04 12:31:52 +02:00
Alexander Tokmakov
e49f9cb655
Merge pull request #38749 from azat/tests-cleanup 
Tiny tests cleanup
 2022-07-04 13:15:15 +03:00
alesapin
be9015852a
Merge pull request #38656 from ClickHouse/some_changes_in_keeper 
Add test for keeper `mntr` command
 2022-07-04 11:00:18 +02:00
zvonand
7d6b7f2612 updated tests 2022-07-04 11:23:37 +03:00
zvonand
79d0206594 updated tests 2022-07-04 11:21:19 +03:00
Alexander Gololobov
612e836e60
Merge pull request #38740 from ClickHouse/array_norm_vectorize 
Improved vectorized execution of main loop for array norm/distance
 2022-07-04 10:19:57 +02:00
Mikhail f. Shiryaev
19516c768f
Revert "Upload to S3 compressed self-extracting clickhouse" 2022-07-04 10:15:27 +02:00
Vitaly Baranov
b4103c1a0e
Merge pull request #38674 from vitlibar/fix-crash-when-grant-all-on-cluster 
Fix crash when granting ALL on cluster.
 2022-07-04 10:13:56 +02:00
Yakov Olkhovskiy
c415543334
Merge pull request #38653 from ClickHouse/deployment-self-extracting 
Upload to S3 compressed self-extracting clickhouse
 2022-07-04 01:45:09 -04:00
Alexey Milovidov
c711012399
Merge pull request #38731 from azat/views-max_insert_threads 
Fix number of threads for pushing to views
 2022-07-04 07:43:26 +03:00
lgbo-ustc
932e596689 Merge remote-tracking branch 'ck/master' into storagehive_alter 2022-07-04 09:56:00 +08:00
zvonand
eac84351f6 fix behavior 2022-07-04 01:26:07 +03:00
Igor Nikonov
2e2ef08712
Merge pull request #37803 from ClickHouse/dictinct_in_order_optimization 
DISTINCT in order optimization
 2022-07-03 21:59:04 +02:00
Alexey Milovidov
bfc9ed6172
Merge pull request #38722 from ClickHouse/fix-rank-corr-functions 
Fix a bug in `rankCorr` function
 2022-07-03 20:30:32 +03:00
Azat Khuzhin
d863f6ce1e tests: add no-backward-compatibility-check for 02351_Map_combinator_dist 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-03 15:20:05 +03:00
Azat Khuzhin
7427adb600 tests: disable 01275_parallel_mv under S3 (since it has thread pool for writes) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-03 15:14:05 +03:00
Azat Khuzhin
9225256dea tests: improve 01275_parallel_mv 
- cover optimize_trivial_insert_select=0/1
- cover max_insert_threads
- convert parallel_view_processing into jinja2

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-03 15:14:05 +03:00
Azat Khuzhin
4ae7db8369 Fix max_insert_threads while pushing to views 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-03 15:14:05 +03:00
Azat Khuzhin
f1eec0b6e8 tests: remove unused stress script 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-03 14:24:35 +03:00
Azat Khuzhin
769017e1f5 tests: move random_str() helper into shell_config.sh 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-03 14:24:24 +03:00
Azat Khuzhin
4375a336fd Fix use-after-free for Map combinator that leads to incorrect result 
This use-after-free can be reproduced with distributed queries.

Also note, that this is not sumMappedArray() and friends (that
previously called sumMap()) but Map combinator.

You will find ASan report in details.

<details>

    READ of size 8 at 0x62d00012d218 thread T186 (QueryPipelineEx)
    2022.07.03 05:09:40.000234 [ 31956 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 1.23 GiB, peak 1.23 GiB, will set to 1.25 GiB (RSS), difference: 19.51 MiB
    2022.07.03 05:09:41.000137 [ 31956 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 1.25 GiB, peak 1.25 GiB, will set to 1.26 GiB (RSS), difference: 3.76 MiB
        #0 0x1233a0d8 in DB::AggregateFunctionSumData<>::get() const build_docker/../src/AggregateFunctions/AggregateFunctionSum.h:245:16
        #1 0x1233a0d8 in DB::AggregateFunctionSum<>::insertResultInto(char*, DB::IColumn&, DB::Arena*) const build_docker/../src/AggregateFunctions/AggregateFunctionSum.h:536:70
        #2 0x1470f910 in DB::AggregateFunctionMap<char8_t>::insertResultInto() const build_docker/../src/AggregateFunctions/AggregateFunctionMap.h:236:26
        #3 0x147110ce in DB::IAggregateFunctionHelper<>::insertResultIntoBatch() const build_docker/../src/AggregateFunctions/IAggregateFunction.h:618:53
        #4 0x2c4269d7 in void DB::Aggregator::convertToBlockImplFinal<>() const build_docker/../src/Interpreters/Aggregator.cpp:1878:49
        #5 0x2c403b9f in void DB::Aggregator::convertToBlockImpl<>() const build_docker/../src/Interpreters/Aggregator.cpp:1714:13
        #6 0x2be09b53 in DB::Aggregator::prepareBlockAndFillSingleLevel() const::$_2::operator()() const build_docker/../src/Interpreters/Aggregator.cpp:2144:9
        #7 0x2be09b53 in DB::Block DB::Aggregator::prepareBlockAndFill<>() const build_docker/../src/Interpreters/Aggregator.cpp:2000:5
        #8 0x2be09b53 in DB::Aggregator::prepareBlockAndFillSingleLevel() const build_docker/../src/Interpreters/Aggregator.cpp:2150:12
        #9 0x2be37de3 in DB::Aggregator::mergeBlocks() build_docker/../src/Interpreters/Aggregator.cpp:3032:17
        #10 0x308c27f8 in DB::MergingAggregatedBucketTransform::transform() build_docker/../src/Processors/Transforms/MergingAggregatedMemoryEfficientTransform.cpp:360:37

    0x62d00012d218 is located 3608 bytes inside of 32768-byte region [0x62d00012c400,0x62d000134400)
    freed by thread T186 (QueryPipelineEx) here:
        #0 0xd701312 in free (/work1/azat/tmp/upstream/clickhouse-asan+0xd701312) (BuildId: b7977aef37e9f720)
        ...
        #8 0x2e3c22eb in DB::ColumnAggregateFunction::~ColumnAggregateFunction() build_docker/../src/Columns/ColumnAggregateFunction.cpp:89:1
        ...
        #18 0xd9fcdd4 in std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> >::~vector() build_docker/../contrib/libcxx/include/vector:401:9
        #19 0x2be373f4 in DB::Aggregator::mergeBlocks() build_docker/../contrib/libcxx/include/__memory/unique_ptr.h
        #20 0x308c27f8 in DB::MergingAggregatedBucketTransform::transform() build_docker/../src/Processors/Transforms/MergingAggregatedMemoryEfficientTransform.cpp:360:37

    previously allocated by thread T186 (QueryPipelineEx) here:
        #0 0xd7015be in malloc (/work1/azat/tmp/upstream/clickhouse-asan+0xd7015be) (BuildId: b7977aef37e9f720)
        #1 0xd85190a in Allocator<false, false>::allocNoTrack(unsigned long, unsigned long) build_docker/../src/Common/Allocator.h:227:27
        #2 0xd988d45 in Allocator<false, false>::alloc(unsigned long, unsigned long) build_docker/../src/Common/Allocator.h:96:16
        #3 0xd988d45 in DB::Arena::MemoryChunk::MemoryChunk(unsigned long, DB::Arena::MemoryChunk*) build_docker/../src/Common/Arena.h:54:64
        #4 0xd98904b in DB::Arena::addMemoryChunk(unsigned long) build_docker/../src/Common/Arena.h:122:20
        #5 0xec9542c in DB::Arena::alignedAlloc(unsigned long, unsigned long) build_docker/../src/Common/Arena.h:171:13
        #6 0x1470f123 in DB::AggregateFunctionMap<char8_t>::deserialize() const build_docker/../src/AggregateFunctions/AggregateFunctionMap.h:205:35

</details>

P.S. Thanks to @den-crane for the reproducer.

Fixes: #35359 (cc @den-crane @dongxiao-yang)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-03 12:36:30 +03:00
mergify[bot]
50c105a6db
Merge branch 'master' into fix-crash-when-grant-all-on-cluster 2022-07-03 09:24:52 +00:00
Alexander Gololobov
8ce8158f7f Do computations in Float32 (not Float64) for arrays of Float32 2022-07-03 10:33:11 +02:00
Alexander Gololobov
ca2829188d Perf test for norm/distance with long arrays of floats 2022-07-03 08:01:49 +02:00
Alexander Gololobov
c6691cc5f2 Improved vectorized execution of main loop for array norm/distance 2022-07-02 22:45:22 +02:00
Alexander Gololobov
92cbc2a3b5 Tests with long arrays 2022-07-02 22:45:22 +02:00
Alexey Milovidov
c4718049c1
Merge pull request #38705 from nickitat/fix_filling_transform 
Fix FillingTransform
 2022-07-02 22:33:44 +03:00
Azat Khuzhin
9b8fc6a5a6 Fix CLICKHOUSE_TMP in tests 
Previous it was not updated, and always uses the first CLICKHOUSE_TMP
for all tests (that was run from one thread).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-02 21:47:29 +03:00
Alexander Gololobov
92fdd62a26
Merge pull request #38545 from israelg99/squared_l2 
Implement L2 Squared Distance and Norm for array and tuples.
 2022-07-02 20:24:54 +02:00
Alexey Milovidov
4ba4329e0d Fix bad test 2022-07-02 19:48:42 +02:00
mergify[bot]
dfac2ca2fc
Merge branch 'master' into fix-crash-when-grant-all-on-cluster 2022-07-02 17:38:45 +00:00
Alexey Milovidov
5574028617
Merge pull request #38728 from azat/expect-tests-debug 
Add exp_internal for expect tests
 2022-07-02 20:25:35 +03:00
alesapin
a1c1b7f7b4
Merge pull request #38703 from ClickHouse/fix-backports-diff 
Fix backports diff
 2022-07-02 18:59:14 +02:00
Azat Khuzhin
e4b81e6de9 Add exp_internal for expect tests 
This is to catch possible issues like 01565_reconnect_after_client_error in [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/38417/e33f236d502fe9f7906392f4ceffaee44e3c4ce4/stateless_tests__thread__actions__[3/3].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-02 17:08:06 +03:00
alesapin
b2db49dbf1
Merge branch 'master' into better_data_part_storage_builder 2022-07-02 15:39:44 +02:00
mergify[bot]
b016be264c
Merge branch 'master' into squared_l2 2022-07-02 09:17:28 +00:00
Alexey Milovidov
16a4b6aa35 Fix a bug in rankCorr function 2022-07-02 03:38:18 +02:00
mergify[bot]
12f5250e86
Merge branch 'master' into dictinct_in_order_optimization 2022-07-01 22:51:35 +00:00
Igor Nikonov
9ef8ff5a31 Addressing review comments 2022-07-01 22:50:00 +00:00
Alexey Milovidov
c2486f3637
Merge pull request #38355 from ClibMouse/add-tlsv1_3-test 
Adding TLS V13 Test
 2022-07-02 01:32:51 +03:00
Yakov Olkhovskiy
776a80bff8
Merge pull request #38136 from bigo-sg/translate_qualified_bug 
Bug fixed in TranslateQualifiedNamesMatcher
 2022-07-01 14:07:00 -04:00
Kruglov Pavel
800baf5299
Fix test 2022-07-01 19:54:43 +02:00
kssenii
6a464c0a24 Merge 2022-07-01 19:12:44 +02:00
kssenii
831d2253ce Merge master 2022-07-01 18:53:20 +02:00
Amos Bird
53f47127e9
Fix only_merge header 2022-07-01 23:31:45 +08:00
Mikhail f. Shiryaev
27bd355525
Use cached image from the release branch for backport PRs 2022-07-01 16:49:27 +02:00
Robert Schulze
2a1ede0f5a
Merge pull request #38589 from ClickHouse/fix-zero-bytes-in-haystack 
Fix countSubstrings() & position() on patterns with 0-bytes
 2022-07-01 16:15:43 +02:00
alesapin
7c53c8bd2c Black 2022-07-01 16:10:06 +02:00
Nikita Taranov
726aec1f51 rm crutch in test 2022-07-01 16:08:26 +02:00
zvonand
3b5332d15e Revert "Revert "Non Negative Derivative window function"" 
This reverts commit dea3b5bfce.
 2022-07-01 18:59:07 +05:00
alesapin
919baf331e Review fixes 2022-07-01 15:57:24 +02:00
Mikhail f. Shiryaev
6a67be3195
Get a proper set of changed files for backport PRs 2022-07-01 15:34:46 +02:00
Mikhail f. Shiryaev
b8fa26f94f
Fix wrong self.changed_files type 2022-07-01 15:34:45 +02:00
Mikhail f. Shiryaev
7cc063a1d0
Fix strange backport titles issues 2022-07-01 12:46:25 +02:00
Amos Bird
84a407f381
Fix toHour monotonicity 2022-07-01 18:24:24 +08:00
Vitaly Baranov
ae2f586170 Fix crash when granting ALL on cluster. 2022-07-01 12:19:56 +02:00
Alexander Tokmakov
f0a5e3aeef
Merge pull request #38662 from ClickHouse/full-queue-stacktraces 
Print stacktraces if test queue is full
 2022-07-01 12:57:00 +03:00
Vitaly Baranov
cadb496894
Merge pull request #38537 from vitlibar/backup-improvements-8 
Backup Improvements 8
 2022-07-01 11:20:46 +02:00
Amos Bird
d73f975849
Fix intermediate header in aggregate projection 2022-07-01 16:47:32 +08:00
Alexey Milovidov
20841f0e1e
Merge pull request #38551 from ClickHouse/revert-37628-non-neg-deriv 
Revert "Non Negative Derivative window function"
 2022-07-01 02:46:28 +03:00
alesapin
0888d6bdb5 Black 2022-07-01 00:43:30 +02:00
Dmitry Novik
7378f0e30f Print stacktraces if test queue is full 2022-06-30 22:33:56 +00:00
alesapin
66705eacb0 Add test for keeper mntr command 2022-07-01 00:07:22 +02:00
Alexander Tokmakov
99e545371b
Merge pull request #38644 from ClickHouse/clean-integration-volumes 
Clean out randomized integration volumes each run
 2022-07-01 00:54:09 +03:00
Yakov Olkhovskiy
12290dd644
upload from self-extracting directory 2022-06-30 17:35:42 -04:00
alesapin
979565edf0 Merge branch 'master' into better_data_part_storage_builder 2022-06-30 22:54:42 +02:00
alesapin
eb5046ab26 Simplify everything 2022-06-30 22:51:27 +02:00
avogar
5b0fd31c64 Put column names in quotes 2022-06-30 16:14:30 +00:00
Mikhail f. Shiryaev
5c5f05dd4b
Clean out randomized integration volumes each run 2022-06-30 17:38:10 +02:00
Alexander Tokmakov
7a1346ad22
Update 02067_lost_part_s3.sql 2022-06-30 18:27:28 +03:00
Mikhail f. Shiryaev
738769d1f7
Download arm64 packages and push to artifactory 2022-06-30 16:31:39 +02:00
Mikhail f. Shiryaev
6925bd6a03
Fix hardcoded retries for get_with_retries 2022-06-30 16:31:37 +02:00
Mikhail f. Shiryaev
e76ebdfb47
Change clickhouse-{client,server} packages architecture 2022-06-30 16:31:31 +02:00
Vitaly Baranov
e367d96964 Fix style. 2022-06-30 15:10:33 +02:00
Igor Nikonov
488ee75fc4 + use DistinctSorted for final distinct step 
+ fix performance tests
 2022-06-30 13:03:39 +00:00
Anton Popov
7c721578c7
Merge pull request #38320 from CurtizJ/dynamic-columns-16 
Improve performace of insertion to columns of type JSON
 2022-06-30 14:18:20 +02:00
Maksim Kita
0de66a2712
Merge pull request #38449 from ClickHouse/revert-38361-revert-38324-fix-partial-sort 
Revert "Revert "Fix optimization in PartialSortingTransform (SIGSEGV and possible incorrect result)""
 2022-06-30 13:02:38 +02:00
Azat Khuzhin
a47355877e
Add revision() function (#38555) 
It can be useful to match versions, since in some tables
(system.trace_log) there is only revision column.

P.S. came to this when was digging into stress reports from CI.
P.P.S. case insensitive by analogy with version().

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-30 12:58:26 +02:00
alesapin
be213c0be7
Merge pull request #38590 from ClickHouse/proper_fix 
Proper fix for ipv4/ipv6 conversion error
 2022-06-30 11:15:01 +02:00
Antonio Andelic
de264117fd
Merge pull request #38118 from bigo-sg/storagehive_struct_type 
Add struct type support in `StorageHive`
 2022-06-30 09:11:13 +02:00
Vitaly Baranov
031ca28fdc Add test for partition clause. More checks for data compatibility on restore. 2022-06-30 08:37:18 +02:00
Vitaly Baranov
11b51d2878 Implement storing UDF in backups. 2022-06-30 08:37:17 +02:00
Vitaly Baranov
efbee5e723 Improve gathering metadata for backup - part 7. 2022-06-30 08:37:17 +02:00
Vitaly Baranov
7689e0c36f Improve gathering metadata for backup - part 6. 2022-06-30 08:37:17 +02:00
Vitaly Baranov
64b51a3772 Improve gathering metadata for backup. 2022-06-30 08:37:17 +02:00
Vitaly Baranov
47ac47350b Store projections in backups. 2022-06-30 08:37:17 +02:00
Alexander Tokmakov
8b4ec60063
Merge pull request #38587 from ClickHouse/nickitat-patch-2 
quick fix for 02112_with_fill_interval
 2022-06-30 02:46:47 +03:00
mergify[bot]
4cbbfb431d
Merge branch 'master' into dictinct_in_order_optimization 2022-06-29 23:32:17 +00:00
Igor Nikonov
d435532c68 Adapt range search algorithm to high cardinality case 
+ range search done in steps of some number of rows.
  Controled by new
  setting `distinct_in_order_range_search_step`. By default 0, i.e.
  whole chunk
+ before start binary search, linear probing is done on each step (32
  rows currently)
 2022-06-29 23:30:35 +00:00
Robert Schulze
81bb2242fd
Fix countSubstrings() & position() on patterns with 0-bytes 
SQL functions countSubstrings(), countSubstringsCaseInsensitive(),
countSubstringsUTF8(), position(), positionCaseInsensitive(),
positionUTF8() with non-const pattern argument use fallback sorters
LibCASCIICaseSensitiveStringSearcher and LibCASCIICaseInsensitiveStringSearcher
which call ::strstr(), resp. ::strcasestr(). These functions assume that
the haystack is 0-terminated and they even document that. However, the
callers did not check if the haystack contains 0-byte (perhaps because
its sort of expensive). As a consequence, if the haystack contained a
zero byte in it's payload, matches behind this zero byte were ignored.

    create table t (id UInt32, pattern String) engine = MergeTree() order by id;
    insert into t values (1, 'x');
    select countSubstrings('aaaxxxaa\0xxx', pattern) from t;

We returned 3 before this commit, now we return 6
 2022-06-29 21:41:18 +00:00
Dmitry Novik
4f7967ab9e
Merge pull request #37887 from azat/union-fix 
Fix incorrect columns order in subqueries of UNION
 2022-06-29 20:40:23 +02:00
alesapin
615070425e Fix comment 2022-06-29 20:30:23 +02:00
alesapin
8f5582f95e Review and style fixes 2022-06-29 20:29:50 +02:00
alesapin
9b387a57ed Test ipv6 as well 2022-06-29 17:56:59 +02:00
Yatsishin Ilya
1ef92b4c27 fix black again 2022-06-29 15:56:29 +00:00
alesapin
62e7a89f26 Proper fix for ipv4/ipv6 conversion error 2022-06-29 17:53:08 +02:00
Maksim Kita
45c18145ec Fixed tests 2022-06-29 17:51:27 +02:00
Nikita Taranov
41460dcaca
quick fix for 02112_with_fill_interval 
Seems like the problem is that now data more often come to FillingTransform in multiple chunks. Don't know why it affects the results, will continue investigation.
 2022-06-29 17:37:29 +02:00
Yatsishin Ilya
f4691b0b00 fix black 2022-06-29 15:08:24 +00:00
Yatsishin Ilya
6f6e814542 Fix test_dictionaries_postgresql 2022-06-29 14:18:12 +00:00
mergify[bot]
36139eacd7
Merge branch 'master' into dictinct_in_order_optimization 2022-06-29 13:37:16 +00:00
Igor Nikonov
3627c6ff36 Perf tests with high cardinality 2022-06-29 13:13:39 +00:00
Yatsishin Ilya
c86c6cc2d9 Improve postgres tests. Fix flaky test_postgresql_database_engine 2022-06-29 12:54:45 +00:00
Anton Popov
c5e0869c46
fix flaky test 2022-06-29 14:20:26 +02:00
Anton Popov
e78814f3bb
Merge pull request #38371 from CurtizJ/fix-distint-with-limit 
Fix `DISTINCT` with `LIMIT` in distributed queries
 2022-06-29 14:02:41 +02:00
Alexander Tokmakov
8246e55002
Merge pull request #38486 from azat/fix-parts-removal 
Fix parts removal after incorrect server shutdown
 2022-06-29 14:17:59 +03:00
mergify[bot]
1280fc39a0
Merge branch 'master' into better-exception-messages 2022-06-29 11:09:06 +00:00
Kruglov Pavel
b9813bcdea
Merge pull request #38356 from kssenii/fix-rabbitmq-with-peekable 
Fix RabbitMQ with formats based on PeekableReadBuffer
 2022-06-29 13:06:10 +02:00
mergify[bot]
9482c99ab8
Merge branch 'master' into sql-insert-format 2022-06-29 11:03:07 +00:00
Alexander Tokmakov
c3573532c0
Merge pull request #38487 from ClickHouse/add_some_tsa_annotations 
Add some TSA annotations
 2022-06-29 13:57:10 +03:00
Mikhail f. Shiryaev
498a9cc8d4
Terminate instance if docker does not start on post-hook 2022-06-29 12:54:33 +02:00
Alexander Tokmakov
ceb66ade4b
Merge pull request #38335 from ClickHouse/deprecate_ordinary_database 
Deprecate Ordinary database and old *MergeTree syntax
 2022-06-29 13:42:59 +03:00
Kruglov Pavel
5cc662f227
Merge pull request #38511 from azat/fix-schema_inference_with_globs 
tests: fix 02305_schema_inference_with_globs flakiness
 2022-06-29 12:39:38 +02:00
Maksim Kita
23e09b5e9b
Merge pull request #35166 from zhangjmruc/sql_compatible_create_drop_index 
Support SQL compatible create/drop index syntax
 2022-06-29 12:30:28 +02:00
Mikhail f. Shiryaev
80e7abd0fe
Merge pull request #38548 from excitoon-favorites/ci-integration-tests-test 
Don't spoil return code of integration tests runner with redundant tee
 2022-06-29 11:18:53 +02:00
Maksim Kita
00372e4646 Fixed tests 2022-06-29 11:05:36 +02:00
Alexander Tokmakov
2aa6c66a2e
Merge pull request #38488 from azat/fix-02335_column_ttl_expired_column_optimization 
tests: cleanup tmp data in 02335_column_ttl_expired_column_optimization
 2022-06-29 12:05:23 +03:00
hexiaoting
e32a0838d1 fix bug for mapUpdate 2022-06-29 15:52:08 +08:00
Alexey Milovidov
0d27672ac8
Merge pull request #38440 from xiedeyantu/allow_trailing_comma 
Allow trailing comma in columns list
 2022-06-29 09:47:09 +03:00
Alexey Milovidov
dea3b5bfce
Revert "Non Negative Derivative window function" 2022-06-29 08:56:15 +03:00
Kseniia Sumarokova
71e9363baf
Fix black 2022-06-29 07:32:27 +02:00
Vladimir Chebotarev
f7ef571842 Don't spoil return code of integration tests runner with redundant tee. 2022-06-29 06:11:09 +03:00
jianmei zhang
d6b0bc2942 Add test case 2022-06-29 11:08:05 +08:00
lgbo-ustc
ee30c4a3f7 update test scripts 2022-06-29 10:23:00 +08:00
lgbo-ustc
7274169c33 update test scripts 2022-06-29 10:03:10 +08:00
Nikita Taranov
f5d26572df
Quick fix for aggregation pipeline (#38295) 2022-06-29 01:16:30 +02:00
Azat Khuzhin
4a52d3f6b9 tests: update reference for 00597_push_down_predicate_long 
CI: https://s3.amazonaws.com/clickhouse-test-reports/37887/6b4613c88056f3901f6d8a832c9e2bc15e22206d/stateless_tests__aarch64__actions_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-28 22:53:03 +03:00
Azat Khuzhin
563fac5333 tests: add a test for incorrect header in UNION queries 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-28 22:53:03 +03:00
Azat Khuzhin
1648cd0eb0 tests: cleanup tmp data in 02335_column_ttl_expired_column_optimization 
v2: remove trap usage as requested by @tavplubix
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-28 22:43:09 +03:00
Julian Gilyadov
d0d72e0b5e
Implement L2Squared Distance and Norm 2022-06-28 15:28:58 -04:00
Maksim Kita
2b11c0daa9 Updated tests 2022-06-28 19:26:15 +02:00
Mikhail f. Shiryaev
e1ac7477c9
Avoid killing terminate-instances task 2022-06-28 17:51:47 +02:00
mergify[bot]
4a1fefb42d
Merge branch 'master' into deprecate_ordinary_database 2022-06-28 15:49:52 +00:00
Alexander Tokmakov
e23f1fa774
Merge pull request #38269 from excitoon-favorites/integrationtestsfixes 
Don't try to kill empty list of containers in `integration/runner` II
 2022-06-28 17:35:48 +03:00
Alexander Tokmakov
4f54abf67f better names for macros 2022-06-28 16:27:18 +02:00
Anton Popov
58c8facebb minor fixes 2022-06-28 14:21:21 +00:00
mergify[bot]
bed1f68c74
Merge branch 'master' into sql-insert-format 2022-06-28 14:20:31 +00:00
Alexander Tokmakov
021bdc335e Merge branch 'master' into add_some_tsa_annotations 2022-06-28 15:42:01 +02:00
Alexander Tokmakov
428628238f Merge branch 'master' into deprecate_ordinary_database 2022-06-28 15:40:33 +02:00
Alexander Tokmakov
b134f53118 fix tests 2022-06-28 14:39:22 +02:00
Alexander Tokmakov
2f45a6ebaa
Update test.py 2022-06-28 15:18:30 +03:00
Alexander Tokmakov
2386554746
Merge pull request #38490 from ClickHouse/test-handle-full-queue 
Handle full queue exception in tests
 2022-06-28 15:03:10 +03:00
Alexander Tokmakov
07cecd4e66
Merge pull request #38516 from ClickHouse/fix_flaky_store_cleanup 
Try to fix flaky test
 2022-06-28 14:56:54 +03:00
kssenii
4c99f608b3 Fix 2022-06-28 13:18:21 +02:00
mergify[bot]
ea416c877f
Merge branch 'master' into better-exception-messages 2022-06-28 11:13:26 +00:00
Kruglov Pavel
ccdeaea2f0
Update 01378_alter_rename_with_ttl_zookeeper.sql 2022-06-28 13:06:49 +02:00
avogar
3b4ecc93bc Fix test and style 2022-06-28 10:48:20 +00:00
Azat Khuzhin
9f147bde39 tests: allow 02305_schema_inference_with_globs to run in parallel 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-28 13:45:55 +03:00
Azat Khuzhin
1f42bf9bb3 tests: fix 02305_schema_inference_with_globs flakiness 
Order of files on disk is not guarantee to match with:
- order of creation
- lexical order

So sometimes 02305_data4.jsonl comes first and 2 rows is enough to get
schema.

Reorganize checks a little to avoid flakiness.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-28 13:45:55 +03:00
Alexander Tokmakov
0c51bd9c23 fix flaky test 2022-06-28 12:38:36 +02:00
Mikhail f. Shiryaev
94f932add1
A tiny improvement in logging 2022-06-28 10:27:25 +02:00
larryluogit
3e48c1d765
Merge branch 'ClickHouse:master' into add-tlsv1_3-test 2022-06-27 22:40:43 -04:00
Alexander Tokmakov
851534c9f7 Merge branch 'master' into deprecate_ordinary_database 2022-06-27 23:47:49 +02:00
Azat Khuzhin
e70cff2cea Fix parts removal after incorrect server shutdown 
Before this patch, and after transaction support had been added #24258,
if the server had not removed old parts on shutdown, then they will left
forever.

Fixes: #24258 (cc @tavplubix)
v2: use Tx::PrehistoricTID over removePartsFromWorkingSet() from ctor
v3: apply black to new test
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-27 23:56:25 +03:00
Dmitry Novik
c24c041b12 Handle full queue exception in tests 2022-06-27 20:54:52 +00:00
Alexander Tokmakov
f4883f1f7e add some TSA aanotations 2022-06-27 22:48:27 +02:00
mergify[bot]
a9c1b68034
Merge branch 'master' into dictinct_in_order_optimization 2022-06-27 20:16:00 +00:00
Vladimir Chebotarev
89dc2844db Fix for integration tests fix. 2022-06-27 23:13:40 +03:00
avogar
9bb68bc6de Add SQLInsert output format 2022-06-27 18:31:57 +00:00
Yakov Olkhovskiy
d5f65ece9b
Merge pull request #38105 from arenadata/ADQM-419 
Add kerberosInit function as a replacement for kinit executable calls in Kafka and HDFS
 2022-06-27 14:19:24 -04:00
Alexander Tokmakov
2283436d24
Merge pull request #38344 from Algunenano/implicit_transactions 
Add implicit_transaction setting
 2022-06-27 18:50:37 +03:00
Alexander Tokmakov
fe1b1aa77b fix my favorite check 2022-06-27 16:16:06 +02:00
Alexander Tokmakov
88dbbcb15d fix tests 2022-06-27 16:10:00 +02:00
avogar
344ddc6e7d Fix tests 2022-06-27 13:55:51 +00:00
Alexander Tokmakov
bec921c9d9
Merge pull request #38265 from ClickHouse/cleanup_garbage_in_store_dir 
Add background cleanup of store/ subdirs
 2022-06-27 16:53:13 +03:00
Kseniia Sumarokova
d1243c1e01
Update test.py 2022-06-27 15:35:35 +02:00
Alexander Tokmakov
7c37079a96 Merge branch 'master' into deprecate_ordinary_database 2022-06-27 14:46:59 +02:00
Mikhail f. Shiryaev
6a67ba91cf
Add public hostname for runner 2022-06-27 14:46:28 +02:00
Mikhail f. Shiryaev
f2611c3707
Refresh team keys for each runner 2022-06-27 14:45:50 +02:00
Mikhail f. Shiryaev
9dffe573bb
Reduce amount of docker logs on the runner 2022-06-27 14:44:59 +02:00
Alexander Tokmakov
c4d815c60b
Update test.py 2022-06-27 15:44:20 +03:00
Anton Kozlov
5f81bcd84f
CLICKHOUSE-1331 Rewrite tuple functions as literals in backwards-compatibility mode (#38096) 
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2022-06-27 14:13:21 +02:00
Mikhail f. Shiryaev
7297e468d5
Migrate team-keys lambda to zip archives deployment 2022-06-27 14:00:32 +02:00
kssenii
2c5aeaaa1a Add auto close for postgres connection 2022-06-27 13:46:52 +02:00
Nikita Taranov
2487ba7f00
Move updateInputStream to ITransformingStep (#37393) 2022-06-27 13:16:52 +02:00
mergify[bot]
da52780cb5
Merge branch 'master' into implicit_transactions 2022-06-27 09:46:34 +00:00
Maksim Kita
3ebe6a03b1
Revert "Revert "Fix optimization in PartialSortingTransform (SIGSEGV and possible incorrect result)"" 2022-06-27 10:37:19 +02:00