thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-09-20 16:50:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
89,855 Commits 8,938 Branches 1,766 Tags 2 GiB
87e896ae05
Commit Graph

18034 Commits

Author SHA1 Message Date
Mikhail f. Shiryaev
3d0e4c56e2
Fix prefetch release branch 2022-05-19 10:05:34 +02:00
Kruglov Pavel
9411406a9c
Update test 2022-05-19 09:25:59 +02:00
Vitaly Baranov
16c8e93dd0
Merge pull request #36941 from nvartolomei/nv/show-grants-current-roles 
Allow a user to inspect grants from granted roles
 2022-05-19 09:20:52 +02:00
avogar
cb8646fbb4 Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-array-map-nothing 2022-05-19 07:18:48 +00:00
Azat Khuzhin
29a8a00656 Add ability to pass QueryKind via clickhouse-client/local (useful for debugging) 
v2: fix LocalConnection::sendQuery() for Suggest (comes w/o client_info) [1]
    [1]: https://s3.amazonaws.com/clickhouse-test-reports/37290/7c85175963226ff78eec542efafcff4e650aa0f0/stateless_tests__ubsan__actions_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-19 07:19:27 +03:00
Vxider
3e05054792 disable parallel test 2022-05-19 01:48:47 +00:00
Michael Nutt
c87638d2ba put hashid behind allow_experimental_hash_functions setting 2022-05-18 19:06:33 -04:00
Azat Khuzhin
dea1706d4e
Fix GROUP BY AggregateFunction (#37093) 
* Fix GROUP BY AggregateFunction

finalizeChunk() was unconditionally converting AggregateFunction to the
underlying type, however this should be done only if the aggregate was
applied.

So pass names of aggregates as an argument to the finalizeChunk()

Fuzzer report [1]:

    Logical error: 'Bad cast from type DB::ColumnArray to DB::ColumnAggregateFunction'. Received signal 6 Received signal Aborted (6)

For the following query:

    SELECT
        arraySort(groupArrayArray(grp_simple)),
        grp_aggreg,
        arraySort(groupArrayArray(grp_simple)),
        b,
        arraySort(groupArrayArray(grp_simple)) AS grs
    FROM data_02294
    GROUP BY
        a,
        grp_aggreg,
        b
    SETTINGS optimize_aggregation_in_order = 1

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/37050/323ae98202d80fc4b311be1e7308ef2ac39e6063/fuzzer_astfuzzerdebug,actions//fuzzer.log

v2: fix conflicts in src/Interpreters/InterpreterSelectQuery.cpp
v3: Fix header for GROUP BY AggregateFunction WITH TOTALS
v4: Add sanity check into finalizeBlock()
v5: Use typeid_cast<&> to get more sensible error in case of bad cast (as suggested by @nickitat)
v6: Fix positions passed to finalizeChunk()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Core/ColumnNumbers.h: remove unused <string>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Optimize finalizeChunk()/finalizeBlock()

v2: s/ByPosition/Mask/ s/by_position/mask/
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-18 23:37:43 +02:00
alesapin
d4abbf54b0
Update 00170_s3_cache.sql 2022-05-18 23:21:07 +02:00
Vitaly Baranov
7c8956ca8d
Merge pull request #36879 from tonickkozlov/tonickkozlov/lazy-marks-loading 
Lazy marks loading
 2022-05-18 22:56:46 +02:00
Robert Schulze
a2214760c6
Remove mark_cache_size from further files 
- changed config.xml/yaml files used by CH's own internal tests which
  are (hopefully) not sensitive to mark_cache_size being set or not

- further occurrences exist but changing them seems a bad idea (e.g.
  because they are in customer-provided data)
 2022-05-18 22:09:14 +02:00
vdimir
2995b69f4a
Swap order of converting_join_columns and before_join steps 2022-05-18 19:31:14 +00:00
Michael Nutt
e453132db8 remove hashid define guard 2022-05-18 15:26:54 -04:00
avogar
cd6a29897e Apply input_format_max_rows_to_read_for_schema_inference for all files in globs in total 2022-05-18 17:56:36 +00:00
Vitaly Baranov
9ccddc44c6
Merge pull request #35767 from azat/ON_CLUSTER-grant 
RFC: Add separate CLUSTER grant
 2022-05-18 19:19:54 +02:00
Maksim Kita
df0cb06209
Merge pull request #37289 from kitaisreal/unary-arithmetic-functions-improve-performance-dynamic-dispatch 
Improve performance of unary arithmetic functions
 2022-05-18 19:16:30 +02:00
Kruglov Pavel
ff59ccd3c7
Update test 2022-05-18 17:43:28 +02:00
Kruglov Pavel
de14e35e47
Fix fast test 2022-05-18 17:16:08 +02:00
Vxider
d10b64468e update code style 2022-05-18 15:00:08 +00:00
avogar
a0369fb9a6 Allow to use String type instead of Binary in Arrow/Parquet/ORC formats 2022-05-18 14:51:21 +00:00
Vxider
04f1b20229 init inner window names 2022-05-18 14:34:40 +00:00
Mikhail f. Shiryaev
610328263c
Merge pull request #37306 from ClickHouse/release-without-prestable 
Release minor and major versions without prestable
 2022-05-18 15:44:42 +02:00
Nikolai Kochetov
a19d4c6f1f
Merge pull request #37138 from azat/integration-tests-iptables 
tests/integration: fix possible race for iptables user rules inside containers
 2022-05-18 14:36:33 +02:00
Mikhail f. Shiryaev
c6e2dd1c43
Create release without draft step 2022-05-18 14:34:41 +02:00
Mikhail f. Shiryaev
1c11823935
Get rid of intermediate prestable release 2022-05-18 14:34:40 +02:00
Mikhail f. Shiryaev
b93a4e2da3
Minor improvements on variables and comments 2022-05-18 14:34:40 +02:00
Mikhail f. Shiryaev
3b1c858bff
Rollback prestable part on failed testing 2022-05-18 14:34:38 +02:00
Mikhail f. Shiryaev
6cedd2f671
Catch all Exceptions in context managers 2022-05-18 14:32:49 +02:00
alesapin
19462bdf9e Drop filesystem cache before test 2022-05-18 14:31:28 +02:00
Anton Popov
715d5b0173
Merge pull request #37270 from Avogar/fix-bool-eof 
Fix Nullable(String) to Nullable(Bool/IPv4/IPv6) conversion
 2022-05-18 14:08:52 +02:00
Vladimir C
7d88b81622
Merge pull request #37193 from azat/fix-limit-by-in-dist-queries 
Fix "Cannot create column of type Set" for distributed queries with LIMIT BY
 2022-05-18 13:28:19 +02:00
Nicolae Vartolomei
32b9959dec Allow a user to inspect grants from granted roles 2022-05-18 10:44:18 +00:00
Vladimir C
1779fb08ed
Merge pull request #36271 from yuuch/support_collation_in_create_query 2022-05-18 12:43:49 +02:00
alesapin
e99c90118e Merge branch 'master' into i_object_storage 2022-05-18 12:36:22 +02:00
avogar
b33ad449a9 Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-defaults 2022-05-18 10:26:38 +00:00
Kruglov Pavel
aaace46da2
Merge pull request #37266 from CurtizJ/fix-mutations-with-object 
Fix mutations in tables with columns of type `Object`
 2022-05-18 12:19:55 +02:00
lgbo-ustc
44d4ada542 fixed code style 2022-05-18 18:15:16 +08:00
Kruglov Pavel
492de1076c
Merge pull request #37232 from zzsmdfj/fix-mysql-datatype-binary0 
fix MySQL database engine to compatible with binary(0) dataType
 2022-05-18 12:15:07 +02:00
Kruglov Pavel
2b655ccb75
Speed up test 00157_cache_dictionary 2022-05-18 12:12:41 +02:00
Mikhail f. Shiryaev
381dab1870
Merge pull request #37301 from ClickHouse/fix-check-labels 
Update run-check.py to match PR template, add comments
 2022-05-18 11:31:48 +02:00
lgbo-ustc
9ed7f2109f fixed black check 2022-05-18 17:02:42 +08:00
lgbo-ustc
afed831785 fixed black check 2022-05-18 14:07:23 +08:00
Bharat Nallan Chakravarthy
c476b8dd92 Merge remote-tracking branch 'upstream/master' into ncb/h3-unidirectionaledges-funcs 2022-05-17 20:10:03 -07:00
lgbo-ustc
0578ef9372 add test case. test_hive_query::test_cache_dir_use 2022-05-18 10:26:12 +08:00
mergify[bot]
37d6da7506
Merge branch 'master' into wv-engine 2022-05-18 01:56:56 +00:00
mergify[bot]
4900d2249f
Merge branch 'master' into wv-populate 2022-05-18 01:56:52 +00:00
Mikhail f. Shiryaev
33eed2280b
Fix linter issues 2022-05-18 01:38:46 +02:00
Mikhail f. Shiryaev
48751f5a91
Checkout master:master for performance output in packager 2022-05-18 01:38:44 +02:00
Mikhail f. Shiryaev
1f67665c95
Build performance output in release builds 2022-05-18 01:38:44 +02:00
Mikhail f. Shiryaev
b929eee8db
Remove redundant CHECK_NAME from build_check.py 2022-05-18 01:38:41 +02:00
mergify[bot]
c33ca1f392
Merge branch 'master' into system-certificates 2022-05-17 19:52:28 +00:00
Mikhail f. Shiryaev
e6e0f0589d
Update run-check.py to match PR template, add comments 2022-05-17 21:43:22 +02:00
mergify[bot]
05305811f8
Merge branch 'master' into fix-bool-eof 2022-05-17 19:28:11 +00:00
mergify[bot]
36b4ed19c5
Merge branch 'master' into unary-arithmetic-functions-improve-performance-dynamic-dispatch 2022-05-17 18:08:24 +00:00
Alexander Gololobov
38f291c70d
Merge pull request #37030 from bharatnc/ncb/h3-missing-traversal-funcs 
add remaining h3 traversal funcs
 2022-05-17 18:19:56 +02:00
zzsmdfj
6a8913d2f6 to compatible with binary(0)-adjust code style 2022-05-17 23:08:40 +08:00
Vxider
0627a3d23a disable parallel tests 2022-05-17 14:24:05 +00:00
avogar
46f4f8a457 Fix use of unitialized memory 2022-05-17 12:59:46 +00:00
zzsmdfj
60ef84bf4c to compatible with binary(0)-fix code style 2022-05-17 20:15:59 +08:00
Maksim Kita
beb34e7062 Improve performance of unary arithmetic functions 2022-05-17 13:53:20 +02:00
Vitaly Baranov
f34a5cdee2
Merge branch 'master' into ON_CLUSTER-grant 2022-05-17 13:21:20 +02:00
Vitaly Baranov
23a61b1596
Merge pull request #36700 from azat/fix-server-reload 
Fix server reload on port change (do not wait for current connections from query context)
 2022-05-17 12:25:21 +02:00
Vxider
e03d6009cc update tests 2022-05-17 08:26:22 +00:00
zzsmdfj
2ba1ca03f4 to compatible with binary(0)-add test 2022-05-17 15:30:38 +08:00
Vxider
70c0adef0b update test 2022-05-17 07:26:52 +00:00
lgbo-ustc
319115cc63 update test case 2022-05-17 11:57:58 +08:00
Vxider
f74c934a61 Merged with master 2022-05-17 02:53:59 +00:00
mergify[bot]
0d0fc23170
Merge branch 'master' into wv-engine 2022-05-17 02:24:18 +00:00
Vxider
3986e82b19 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into wv-populate 2022-05-17 02:18:17 +00:00
Vxider
f50803b0b8 update test 2022-05-17 02:18:09 +00:00
Kseniia Sumarokova
94683786dc
Merge branch 'master' into MeiliSearch 2022-05-16 22:42:09 +02:00
Vitaly Baranov
72bff5242a
Merge pull request #37168 from vitlibar/backup-improvements-5 
Backups Improvements 5
 2022-05-16 21:25:54 +02:00
Alexander Gololobov
e2e3536a80 Fixed handling of gridPathCellsSize() errors 2022-05-16 21:23:45 +02:00
avogar
415aabd4d0 Fix Nullable(String) to Nullable(Bool/IPv4/IPv6) conversion 2022-05-16 19:15:18 +00:00
Anton Popov
b6c5ab4fcf fix mutations in tables with columns of type Object 2022-05-16 18:26:53 +00:00
Vxider
d7dcb1f5d9 update test 2022-05-16 17:49:14 +00:00
Yakov Olkhovskiy
863c52f0f4 refactoring, test added 2022-05-16 13:31:28 -04:00
alesapin
ce6d5f7dcc Merge branch 'master' into i_object_storage 2022-05-16 17:42:22 +02:00
Dmitry Novik
dd1e7b55b8
Merge pull request #37050 from azat/fix-optimize_aggregation_in_order-prefix-Array 
Fix optimize_aggregation_in_order with prefix GROUP BY and *Array aggregate functions
 2022-05-16 17:17:56 +02:00
Anton Popov
9c3daf7e5b
Merge pull request #37245 from ClickHouse/CurtizJ-patch-4-1 
Temporarily fix flaky test `01825_type_json_insert_select.sql`
 2022-05-16 16:44:27 +02:00
avogar
2d4b4b9008 Fix inserting defaults for missing values in columnar formats 2022-05-16 14:19:44 +00:00
Azat Khuzhin
367daca584 Add a test for SYSTEM RELOAD CONFIG directly from the query context 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-16 16:22:58 +03:00
Azat Khuzhin
568e29ceb6 tests/integration/test_server_reload: add pylint hints 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-16 16:22:58 +03:00
Azat Khuzhin
b528a72e89 tests/integration/test_server_reload: add missing assertions 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-16 16:22:58 +03:00
Kruglov Pavel
5e34f48a18
Merge pull request #37021 from excitoon-favorites/fixtdigest 
Fixed problem with infs in `quantileTDigest`
 2022-05-16 15:21:59 +02:00
mergify[bot]
b7d6d97e64
Merge branch 'master' into wv-alter 2022-05-16 12:44:46 +00:00
mergify[bot]
753fac95ef
Merge branch 'master' into wv-watch-limit 2022-05-16 11:42:31 +00:00
alesapin
1829256ae2
Merge pull request #37178 from ClickHouse/ignore_auto_increment_in_create_table_2 
Ignore AUTO_INCREMENT in CREATE TABLE to simplify migration from MySQL
 2022-05-16 13:29:22 +02:00
Anton Popov
848b762eb0 Update 01825_type_json_insert_select.sql 2022-05-16 11:18:31 +00:00
Azat Khuzhin
01e1c5345a Add separate CLUSTER grant 
In case you have different roles for the same user on multiple clusters,
ON CLUSTER query can help to overcome some limitations.

Consider the following example:
- cluster_with_data, dev_user (readonly=2)
- stage_cluster, dev_user (readonly=0)

So when you will execute the following query from stage_cluster, it will
be successfully executed, since ON CLUSTER queries has different system
profile:

    DROP DATABASE default ON CLUSTER cluster_with_data

This is not 100% safe, but at least something.

Note, that right now only ON CLUSTER query it self is supported, but
separate clusters are not (i.e. GRANT CLUSTER some_cluster_name TO
default), since right now grants sticked to database+.

v2: on_cluster_queries_require_cluster_grant
v3: fix test and process flags as bit mask
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-16 13:57:45 +03:00
Vladimir C
75008d5903
Merge pull request #37166 from vdimir/issue_37141 2022-05-16 12:21:41 +02:00
Anton Ivashkin
191767d300 Reformat test file according to Check black 2022-05-16 12:35:21 +03:00
Vladimir Chebotarev
5bbab401c4 Added test. 2022-05-15 22:49:52 +03:00
Vxider
af3e13541a update test 2022-05-15 17:56:53 +00:00
Vxider
ba472c97b2 update tests 2022-05-15 17:48:01 +00:00
Vxider
ea62dc4d0a use event time in test 2022-05-15 16:05:19 +00:00
Dmitry Novik
a36f05d6b8
Merge pull request #32999 from ClickHouse/fix-gs-formating 
Cleanup GROUPING SETS formating
 2022-05-15 15:11:52 +02:00
Dmitry Novik
2259add1fa
Merge pull request #37197 from ClickHouse/overcommit-ratio-overflow 
Fix possible overflow in OvercommitRatio
 2022-05-15 15:09:19 +02:00
Vitaly Baranov
c1baad0763 Fix style. 2022-05-15 14:09:42 +02:00
Vxider
3f64814544 update test 2022-05-15 11:37:40 +00:00
Vxider
528e21ebf4 add test for windowview watch limit 2022-05-15 09:14:01 +00:00
Alexey Milovidov
3b9d8f9d19
Merge pull request #37095 from ClickHouse/upload-ppc64le-binary 
Add PowerPC binary to universal install script
 2022-05-15 04:33:06 +03:00
Alexey Milovidov
8334329dff
Merge pull request #37023 from ClickHouse/chained-or 
Merging #34932.
 2022-05-15 04:32:31 +03:00
Vxider
d7f2be1b29 update test 2022-05-14 17:49:51 +00:00
alesapin
4e7e304e3d Fix BLACK 2022-05-14 18:53:30 +02:00
Vxider
3146308ab5 add test 2022-05-14 16:47:47 +00:00
Vxider
f459dd5aba change windowview engine behaviour 2022-05-14 16:46:49 +00:00
alesapin
61014dbde6 Simlify strange test 2022-05-14 18:41:54 +02:00
alesapin
32e0ca22be Format queries better 2022-05-14 14:49:01 +02:00
alesapin
4540cf8925 Style check fixes 2022-05-14 13:46:47 +02:00
Vxider
7802c0d7d9 update test 2022-05-14 11:36:22 +00:00
Vxider
f8a68fff5b update test 2022-05-14 09:39:55 +00:00
Azat Khuzhin
40f3dbb448 Ignore harmful env variables in clickhouse binaries (reexec w/o them) 
Some environments may really require LD_LIBRARY_PATH (and some other
variables), so rejecting running clickhouse binaries in such envs is a
backward incompatible change.

So instead of rejecting, let's ignore those env variables, i.e. reexec
binaries without them.

Also note, that there is no messages in stderr in case of some of
variables set anymore, since this message may break some scripts.

Refs: #36340
Follow-up for: #36342
 2022-05-14 12:08:48 +03:00
Vxider
46f1a59f3d update test 2022-05-14 06:36:08 +00:00
Vxider
d1a9ebaaaf update code style 2022-05-14 03:59:23 +00:00
mergify[bot]
4e184306c1
Merge branch 'master' into wv-alter 2022-05-14 02:11:46 +00:00
Michael Nutt
8bff9b8ce9
Merge branch 'master' into hashid 2022-05-14 09:52:05 +09:00
Mikhail f. Shiryaev
8a1cc46212
Merge pull request #37203 from DevTeamBK/fix_coverity 
Fix coverity build problem with LD_PRELOAD
 2022-05-14 00:42:21 +02:00
alesapin
cc304a3178 Merge branch 'master' into i_object_storage 2022-05-13 23:44:00 +02:00
Maksim Kita
3f18d7da33
Merge pull request #37189 from kitaisreal/function-h3-k-ring-add-cast 
Function h3kRing added cast
 2022-05-13 22:53:20 +02:00
Vxider
3bdddea3cf update test 2022-05-13 17:45:28 +00:00
bkuschel
16e4b0db20
Fix coverity build 2022-05-13 10:22:23 -07:00
Azat Khuzhin
b29a0f3c84 Fix "Cannot create column of type Set" for distributed queries with LIMIT BY 
The problem was that after #36454 it requires too much columns.

Fixes: #36454
Fixes: #37045
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-13 20:03:19 +03:00
Dmitry Novik
75c5ae31f7 Fix shellcheck 2022-05-13 16:27:52 +00:00
Dmitry Novik
fb5766287c Cleanup GROUPING SETS formating 2022-05-13 16:14:41 +00:00
Dmitry Novik
45585f0e7f Fix possible overflow in OvercommitRatio 2022-05-13 15:39:05 +00:00
Alexander Gololobov
fe2aa1861f
Merge pull request #37033 from qieqieplus/array-distance-functions 
Distance/ Norm functions for array
 2022-05-13 17:28:19 +02:00
avogar
cef13c2c02 Allow to skip unknown columns in Native format 2022-05-13 14:27:15 +00:00
Mikhail Artemenko
c14fae24c4 run black for python files 2022-05-13 16:55:42 +03:00
avogar
b17fec659a Improve performance and memory usage for select of subset of columns for some formats 2022-05-13 13:51:28 +00:00
Vxider
930939ef75 update code style 2022-05-13 13:38:53 +00:00
Igor Nikonov
74be0946ff Test fixes 2022-05-13 13:33:45 +00:00
Vxider
273827187c update test 2022-05-13 13:26:38 +00:00
mergify[bot]
603bd85fbe
Merge branch 'master' into function-h3-k-ring-add-cast 2022-05-13 13:26:09 +00:00
Maksim Kita
ef7e21ea46 Function h3kRing added cast 2022-05-13 15:20:04 +02:00
Vxider
121fd26ab1 add test 2022-05-13 13:10:34 +00:00
mergify[bot]
4a661b6e78
Merge branch 'master' into json-columns-formats 2022-05-13 11:32:03 +00:00
Igor Nikonov
8d5cabc8f7 Fix style warnings and review comments 2022-05-13 11:27:17 +00:00
mergify[bot]
756a8520e9
Merge branch 'master' into fix-array-map-nothing 2022-05-13 10:45:36 +00:00
vdimir
dc2a90a344
ApplyWithGlobalVisitor for WITH in subquery 2022-05-13 10:40:51 +00:00
Kseniia Sumarokova
6d5137879b
Merge pull request #37054 from kssenii/better-read-from-cache 
Better read from cache buffer
 2022-05-13 12:20:06 +02:00
Alexander Tokmakov
432f1d570c
Merge pull request #37153 from azat/fix-optimize_skip_unused_shards_rewrite_in 
Fix implicit cast for optimize_skip_unused_shards_rewrite_in
 2022-05-13 12:58:28 +03:00
Igor Nikonov
f423e9bf40 Ignore AUTO_INCREMENT in CREATE TABLE to simplify migration from MySQL 
Issue #35893
 2022-05-13 09:22:52 +00:00
Azat Khuzhin
b0c72b6104 tests/integration: add prefix match for skipped tests 
This way you can specify only file/module, or test name without
parameters.

Since, at least one, test that we care about
(test_distributed_respect_user_timeouts/test.py::test_reconnect) was not
runned sequentially [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/37138/d7f5db4143c559bb8044058a653956945897e724/integration_tests__asan__actions__[2/3].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-13 11:19:11 +03:00
Azat Khuzhin
f886c4237e tests/integration: fix possible race for iptables user rules inside containers 
It is possible for network PartitionManager to work incorrectly, because
of how docker setting up forward to DOCKER-USER chain, it first removes
forward and then adds it back (see [1] and [2]), however this introduce
race for a short period of time, and this is enough for TCP to
retransmit packets, and breaks network PartitionManager.

  [1]: b1e30e8328/libnetwork/iptables/iptables.go (L638)
  [2]: b1e30e8328/libnetwork/firewall_linux.go (L42)

Here are some details from logs for [3]:

    2022-04-27 03:01:00 [ 621 ] DEBUG : Executing query SELECT node FROM distributed_table ORDER BY node on node2 (cluster.py:2879, query_and_get_error)

  [3]: https://s3.amazonaws.com/clickhouse-test-reports/36295/314d553ab14d30df7508814513506ec09c7c7061/integration_tests__asan__actions__[2/3]/integration_run_parallel1_0.log

This query fails, from the server logs:

    2022.04.27 03:01:00.213101 [ 10 ] {19b1719f-8c39-4e3e-b782-aa4c933650f2} <Debug> executeQuery: (from 172.16.5.1:59008) SELECT node FROM distributed_table ORDER BY node
    ...
    2022.04.27 03:01:03.578439 [ 223 ] {19b1719f-8c39-4e3e-b782-aa4c933650f2} <Debug> Connection (node1:9000): Sent data for 2 scalars, total 2 rows in 0.000284672 sec., 6993 rows/sec., 68.00 B (232.15 KiB/sec.), compressed 0.4594594594594595 times to 148.00 B (505.16 KiB/sec.)
    2022.04.27 03:01:03.590637 [ 223 ] {19b1719f-8c39-4e3e-b782-aa4c933650f2} <Debug> MergingSortedTransform: Merge sorted 3 blocks, 2 rows in 3.371592744 sec., 0.5931914533744174 rows/sec., 94.61 B/sec
    2022.04.27 03:01:03.601256 [ 10 ] {19b1719f-8c39-4e3e-b782-aa4c933650f2} <Information> executeQuery: Read 2 rows, 28.00 B in 3.387950542 sec., 0 rows/sec., 8.26 B/sec.
    2022.04.27 03:01:03.601894 [ 10 ] {19b1719f-8c39-4e3e-b782-aa4c933650f2} <Debug> MemoryTracker: Peak memory usage (for query): 334.38 KiB.

And from docker daemon log:

    time="2022-04-27T03:00:59.916693113Z" level=debug msg="form data: {\"AttachStderr\":true,\"AttachStdin\":false,\"AttachStdout\":true,\"Cmd\":[\"iptables\",\"--wait\",\"-I\",\"DOCKER-USER\",\"1\",\"-p\",\"tcp\",\"-s\",\"172.16.5.2\",\"-d\",\"172.16.5.3\",\"-j\",\"DROP\"],\"Container\":\"b75f3b68cda51386bfbb9cceb67e92c4d217a5a1660bde2470b583cb1f4c7fc4\",\"Privileged\":true,\"Tty\":false,\"User\":\"\"}"
    time="2022-04-27T03:01:00.030654116Z" level=debug msg="form data: {\"AttachStderr\":true,\"AttachStdin\":false,\"AttachStdout\":true,\"Cmd\":[\"iptables\",\"--wait\",\"-I\",\"DOCKER-USER\",\"1\",\"-p\",\"tcp\",\"-s\",\"172.16.5.3\",\"-d\",\"172.16.5.2\",\"-j\",\"DROP\"],\"Container\":\"b75f3b68cda51386bfbb9cceb67e92c4d217a5a1660bde2470b583cb1f4c7fc4\",\"Privileged\":true,\"Tty\":false,\"User\":\"\"}"
    ...
    time="2022-04-27T03:01:03.515813984Z" level=debug msg="/usr/sbin/iptables, [--wait -t filter -n -L DOCKER-USER]"
    time="2022-04-27T03:01:03.531106486Z" level=debug msg="/usr/sbin/iptables, [--wait -t filter -C DOCKER-USER -j RETURN]"
    time="2022-04-27T03:01:03.535442346Z" level=debug msg="/usr/sbin/iptables, [--wait -t filter -C FORWARD -j DOCKER-USER]"
    time="2022-04-27T03:01:03.555856911Z" level=debug msg="/usr/sbin/iptables, [--wait -D FORWARD -j DOCKER-USER]"
    time="2022-04-27T03:01:03.564905764Z" level=debug msg="/usr/sbin/iptables, [--wait -I FORWARD -j DOCKER-USER]"
    ...
    time="2022-04-27T03:01:03.706374466Z" level=debug msg="form data: {\"AttachStderr\":true,\"AttachStdin\":false,\"AttachStdout\":true,\"Cmd\":[\"iptables\",\"--wait\",\"-D\",\"DOCKER-USER\",\"-p\",\"tcp\",\"-s\",\"172.16.5.3\",\"-d\",\"172.16.5.2\",\"-j\",\"DROP\"],\"Container\":\"b75f3b68cda51386bfbb9cceb67e92c4d217a5a1660bde2470b583cb1f4c7fc4\",\"Privileged\":true,\"Tty\":false,\"User\":\"\"}"
    time="2022-04-27T03:01:03.968077970Z" level=debug msg="form data: {\"AttachStderr\":true,\"AttachStdin\":false,\"AttachStdout\":true,\"Cmd\":[\"iptables\",\"--wait\",\"-D\",\"DOCKER-USER\",\"-p\",\"tcp\",\"-s\",\"172.16.5.2\",\"-d\",\"172.16.5.3\",\"-j\",\"DROP\"],\"Container\":\"b75f3b68cda51386bfbb9cceb67e92c4d217a5a1660bde2470b583cb1f4c7fc4\",\"Privileged\":true,\"Tty\":false,\"User\":\"\"}"

I've tried multiple ways of fixing this:

- Creating separate chain for rules from PartitionManager (DOCKER-USER-CLICKHOUSE)
  But it is created only once, and docker places new rules on top of the
  FORWARD chain, so it will not work, since it will not receive any
  packets

- Use DOCKER-USER, but replace iptables with a wrapper ([script]), that
  will ignore recreating of a rule for forward to DOCKER-USER, but this
  will not work too, since new docker rules will be created on top of
  FORWARD chain, and so DOCKER-USER will packets.

  [script]:

    if [[ "$*" =~ "-D FORWARD -j DOCKER-USER" ]]; then
        exit 0
    fi
    if [[ "$*" =~ "-I FORWARD -j DOCKER-USER" ]]; then
        if iptables.real iptables -C FORWARD -j DOCKER-USER; then
            exit 0
        fi
    fi

- And the only way to avoid flakiness for this case, is to forbid
  parallel execution for tests with PartitionManager.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-13 10:57:24 +03:00
Azat Khuzhin
9899b3f92d tests/integration: remove superfluous import of PartitionManager 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-13 10:57:24 +03:00
Azat Khuzhin
0fb9553e1c tests/integration: use no-resolve and verbose for iptables --list 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-13 10:57:24 +03:00
Azat Khuzhin
5c1b0f3a76 tests/integration: add logging for NetworkManager 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-13 10:57:24 +03:00
mergify[bot]
2fdd305ef1
Merge branch 'master' into array-distance-functions 2022-05-13 07:56:57 +00:00
bharatnc
5cdd577153 fix tests 2022-05-12 12:13:37 -07:00
Kseniia Sumarokova
6e0a910fcd
Merge branch 'master' into better-read-from-cache 2022-05-12 19:44:18 +02:00
Vitaly Baranov
23322b0bf6 Add async tests. 2022-05-12 19:42:05 +02:00
Kruglov Pavel
3a1f6f4fb2
Delete 02295_type_nothing.sql 2022-05-12 18:59:48 +02:00
Kruglov Pavel
d1b5362250
Fix tests 2022-05-12 18:59:26 +02:00
mergify[bot]
5856856884
Merge branch 'master' into http_proper_summary_and_exception 2022-05-12 16:43:25 +00:00
Vladimir C
182ba1c605
Merge pull request #36928 from Vxider/wv-fix-multiple-columns 2022-05-12 18:28:15 +02:00
alesapin
e7296a2b28
Merge pull request #36825 from kssenii/cache-fix-1 
Removed forceful drop cache command, fix detached status state
 2022-05-12 17:18:21 +02:00
avogar
0311dbb422 Add default implementation for Nothing, support arrays of nullable for arrayFilter and similar functions 2022-05-12 15:15:31 +00:00
Vxider
dc33c3a9bb Merged with master 2022-05-12 23:04:07 +08:00
Kseniia Sumarokova
9fb2df2456
Merge pull request #37105 from Vxider/wv-read 
Support SELECT query in WindowView
 2022-05-12 16:52:28 +02:00
Mikhail Artemenko
12717f2417 fix <Breaking changes> in Meilisearch v0.27.0 2022-05-12 16:51:00 +03:00
Azat Khuzhin
ef921d834d tests: improve 01756_optimize_skip_unused_shards_rewrite_in 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-12 16:17:30 +03:00
Azat Khuzhin
e3ff630612 tests: add echo for 01756_optimize_skip_unused_shards_rewrite_in 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-12 16:17:30 +03:00
Azat Khuzhin
586eda5242 Fix implicit cast for optimize_skip_unused_shards_rewrite_in 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Fixes: #37073
 2022-05-12 16:17:30 +03:00
Vladimir C
4cb6ca2311
Merge pull request #36970 from ClickHouse/concurrent_join 
Follow up for ConcurrentHashJoin
 2022-05-12 15:09:29 +02:00
kssenii
483e056bcc Merge master 2022-05-12 14:58:24 +02:00
Vitaly Baranov
2c92fe21a9 Implement restoring to a bigger or smaller cluster. 
Remove backup settings allow_storing_multiple_replicas: now it's always allowed.
 2022-05-12 14:55:06 +02:00
Dmitry Novik
df071bb701
Merge pull request #37129 from ClickHouse/enable-overcommit-for-user 
Update documentation and defaults for memory overcommit
 2022-05-12 14:38:43 +02:00
Alexander Gololobov
669637108c Added tests for special cases with float/double 2022-05-12 14:28:29 +02:00
mergify[bot]
7ffad69019
Merge branch 'master' into wv-populate 2022-05-12 11:52:04 +00:00
Vitaly Baranov
30005a1eff BACKUP ON CLUSTER correctly collects data of a replicated table from all replicas now, 
and if some part doesn't exist on either replica it's no problem anymore.
 2022-05-12 13:33:42 +02:00
Kruglov Pavel
080a44f7a2
Update test 2022-05-12 13:29:25 +02:00
Kruglov Pavel
9125b95aa2
Delete out 2022-05-12 13:28:55 +02:00
Alexander Gololobov
8e843a1e7c
Merge pull request #37077 from ClickHouse/disable-log-query-threads 
Disable `log_query_threads` by default
 2022-05-12 13:12:55 +02:00
Kseniia Sumarokova
59bfcd917b
Merge pull request #36876 from kssenii/extend-cache-log 
Log into filesystem_cache_log when cache not even attempted
 2022-05-12 12:38:36 +02:00
Michael Nutt
4eb55d17b3 consolidate hashid test queries and hold out from fasttest 2022-05-12 19:27:58 +09:00
Nikolai Kochetov
55188665f4
Merge pull request #37071 from ClickHouse/join_dup_columns_issue_contin 
Continue fixing #36199.
 2022-05-12 11:36:15 +02:00
Mikhail f. Shiryaev
373e329525
Merge pull request #37143 from ClickHouse/fasttest-ccache-permissions 
Fix fasttest ccache permissions
 2022-05-12 11:23:58 +02:00
Maksim Kita
9b354268fa
Merge pull request #34469 from kitaisreal/jit-compile-sort-description 
Added JIT compilation of SortDescription
 2022-05-12 10:14:21 +02:00
Robert Schulze
f8c24c5fe8
Merge pull request #37117 from ClickHouse/bug-37114 
Fix Bug 37114 - ilike on FixedString(N)s produces wrong results
 2022-05-12 09:39:36 +02:00
Mikhail f. Shiryaev
e9733f0cbe
Fix fasttest ccache permissions 2022-05-12 09:18:28 +02:00
mergify[bot]
c9cb5496cd
Merge branch 'master' into wv-read 2022-05-12 01:57:34 +00:00
lgbo-ustc
5115fa8c6f update test case tests/integration/test_hive_query/test.py 2022-05-12 09:32:21 +08:00
alesapin
5dcd25be23 Initial implementation 2022-05-12 00:04:54 +02:00
Kseniia Sumarokova
3d35e26e18
Update test 2022-05-11 22:26:38 +02:00
mergify[bot]
f4bae3969d
Merge branch 'master' into enable-overcommit-for-user 2022-05-11 20:04:20 +00:00
Maksim Kita
437d70d4da Fixed tests 2022-05-11 21:59:51 +02:00
Maksim Kita
837f2e8b9c Update performance test 2022-05-11 21:59:51 +02:00
Maksim Kita
ea8ce3140a Fixed tests 2022-05-11 21:59:51 +02:00
Maksim Kita
d85d72e5ad Added performance tests 2022-05-11 21:59:51 +02:00
Nikolai Kochetov
ec34761d9f
Merge pull request #33631 from ClickHouse/grouping-sets-fix 
Support GROUPING SETS
 2022-05-11 21:28:46 +02:00
Michael Nutt
2ff13c4e5d
Merge branch 'master' into hashid 2022-05-12 03:12:10 +09:00
Michael Nutt
0d1d6fb27b consolidate hashid test queries and hold out from fasttest 2022-05-12 02:19:05 +09:00
Dmitry Novik
17608b3d93 Update documentation and defaults for memory overcommit 2022-05-11 16:18:41 +00:00
Kruglov Pavel
af3db5a992
Merge pull request #36726 from PolyProgrammist/s3clustermacro 
Handle cluster macro in s3Cluster table function
 2022-05-11 18:00:57 +02:00
Alexander Gololobov
b34a55c9e9
Merge branch 'master' into array-distance-functions 2022-05-11 16:55:02 +02:00
Yakov Olkhovskiy
6d3a54a044
Merge pull request #36467 from olevino/wyhash 
Wyhash
 2022-05-11 09:57:09 -04:00
Kseniia Sumarokova
567c9e6226
Update 02286_drop_filesystem_cache.reference 2022-05-11 15:55:12 +02:00
Kruglov Pavel
8af9ab3766
Fix test 2022-05-11 15:02:23 +02:00
kssenii
c5e84950d6 Update test, add descriptions to profile events 2022-05-11 14:57:51 +02:00
Robert Schulze
7232f47c68
Fix Bug 37114 - ilike on FixedString(N) columns produces wrong results 
The main fix is in MatchImpl.h where the "case_insensitive" parameter is
added to Regexps::get().

Also made "case_insensitive" a non-default template parameter to reduce
the risk of future bugs.

The remainder of this commit are minor random code improvements.

resoves #37114
 2022-05-11 14:30:21 +02:00
Kseniia Sumarokova
28630b2535
Update test 2022-05-11 14:13:21 +02:00
Alexander Tokmakov
c604fb1a71
Merge pull request #37101 from azat/stateless-tests-fixes 
tests: fix 01119_optimize_trivial_insert_select (due to max_threads randomization)
 2022-05-11 14:15:28 +03:00
Kruglov Pavel
f664b6a5be
Merge pull request #37043 from Avogar/fix-arrow-dict 
Fix LowCardinality->ArrowDictionary invalid output
 2022-05-11 13:01:53 +02:00
avogar
246aafa58a Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-array-map-nothing 2022-05-11 10:51:14 +00:00
Nikolai Kochetov
4b8a2e2d80 Fix fuzzed queries. 2022-05-11 10:22:34 +00:00
Anton Kozlov
23e6792898 Lazy marks loading 2022-05-11 10:15:56 +00:00
Kseniia Sumarokova
d4823eed12
Update 02240_system_remote_filesystem_cache_log.reference 2022-05-11 12:09:16 +02:00
Kseniia Sumarokova
cc5eea4868
Update 02240_system_remote_filesystem_cache_log.sql 2022-05-11 12:08:41 +02:00
Vxider
88930ae42f add test 2022-05-11 09:01:12 +00:00
Mikhail f. Shiryaev
3fa955aec1
Merge pull request #37057 from ClickHouse/changelog 
Changelog script
 2022-05-11 10:54:29 +02:00
mergify[bot]
0e2a86dcee
Merge branch 'master' into MeiliSearch 2022-05-11 08:49:19 +00:00
Mikhail f. Shiryaev
d5b1e39bb9
Merge pull request #36866 from ClickHouse/integration-tests 
Fix using proper image for integration tests; improve logging; fix docker volume cleaning in integration-tests-runner.
 2022-05-11 10:46:51 +02:00
kssenii
18dd74be0c Remove unneede changes 2022-05-11 10:26:30 +02:00
qieqieplus
5f9eee976f fix & format 2022-05-11 16:14:43 +08:00
Azat Khuzhin
91042ff4eb tests: fix 01119_optimize_trivial_insert_select (due to max_threads randomization) 2022-05-11 07:25:31 +03:00
Alexey Milovidov
b793bff179 Update test 2022-05-11 03:09:53 +02:00
Alexey Milovidov
99cc8d47fe Add PowerPC binary to universal install script 2022-05-11 03:02:01 +02:00
Alexey Milovidov
4f99b85c96 Added a test for aliases 2022-05-11 01:42:53 +02:00
Alexey Milovidov
ad9a377f42 Add settings randomization 2022-05-11 01:30:59 +02:00
Alexey Milovidov
3fcf63830c Merge branch 'master' into chained-or 2022-05-11 01:28:12 +02:00
Alexey Milovidov
bde2733aa2 Update tests 2022-05-11 01:25:41 +02:00
alesapin
0f276a0956
Merge pull request #36947 from ClickHouse/dump_ordered_sessions_map 
Force keeper snapshot equality on different replicas
 2022-05-10 23:36:09 +02:00
Nikolai Kochetov
b6075031d8 Delete GroupingSetsTransform. 2022-05-10 17:54:36 +00:00
kssenii
ccab49e3cf Merge master 2022-05-10 19:33:34 +02:00
Kruglov Pavel
71edcc8ab6
Fix fasttest 2022-05-10 18:32:20 +02:00
avogar
02679c7222 Fix tests 2022-05-10 16:27:59 +00:00
avogar
ea0362b3a3 Fix tests 2022-05-10 16:20:38 +00:00
kssenii
b654eda2b4 Better test, remove file 2022-05-10 18:17:22 +02:00
Mikhail f. Shiryaev
76c3022029
Fix code broken in 1e82b04de3 2022-05-10 17:41:47 +02:00
Mikhail f. Shiryaev
eb219ac014
Improve logging in integration_test_check.py 2022-05-10 17:41:01 +02:00
vdimir
cadd614ce8
Add cases to join_dup_columns_issue36199 2022-05-10 14:39:01 +00:00
vdimir
837d4bd13f
Revert "Continue fixing #36199." 
This reverts commit 1feba64a4e.
 2022-05-10 14:30:02 +00:00
Nikolai Kochetov
1feba64a4e
Continue fixing #36199. 2022-05-10 14:26:49 +00:00
avogar
f6b16880bd Merge branch 'master' of github.com:ClickHouse/ClickHouse into json-columns-formats 2022-05-10 12:57:18 +00:00
Anton Popov
f075070fdf
Merge pull request #37046 from azat/fix-optimize_aggregation_in_order-Array 
Fix optimize_aggregation_in_order with *Array aggregate functions
 2022-05-10 14:01:01 +02:00
Anton Popov
aec30c4076
Merge pull request #37053 from CurtizJ/remove-streams-comments 
Remove last mentions of data streams
 2022-05-10 13:38:13 +02:00
alesapin
dcf2a1d245 Turn off gcc-11 2022-05-10 13:24:20 +02:00
alesapin
41737d66b0
Merge pull request #37014 from ClickHouse/better_delete_outdated_parts 
Fix forgotten parts in cleanup thread
 2022-05-10 12:34:34 +02:00
Alexander Tokmakov
a3d922d6f6
Merge pull request #37047 from ClickHouse/fix_trivial_insert_select 
Resubmit forgotten patch
 2022-05-10 13:02:05 +03:00
Nikolai Kochetov
a3df693ace
Merge pull request #37040 from ClickHouse/selected_rows_and_merges 
Do not increase SelectedRows and SelectedBytes in merges and mutations.
 2022-05-10 12:01:06 +02:00
Mikhail f. Shiryaev
24b7a7538d
Add kwargs to runner and is_shallow function 2022-05-10 11:25:33 +02:00
Alexander Tokmakov
9e94105454
Update 01119_optimize_trivial_insert_select.reference 2022-05-10 11:57:16 +03:00
Alexander Tokmakov
702122a502
Update 01119_optimize_trivial_insert_select.sql 2022-05-10 11:55:44 +03:00
Alexander Tokmakov
1d711496c3
Merge pull request #37035 from ClickHouse/Avogar-patch-4 
Fix flaky integration mongodb test
 2022-05-10 11:50:26 +03:00
Vxider
bb6b1883fc add tests for windowview populate 2022-05-10 15:07:23 +08:00
bharatnc
592de6895c Merge remote-tracking branch 'upstream/master' into ncb/h3-missing-traversal-funcs 2022-05-09 22:41:03 -07:00
bharatnc
f8d54b514f fix tests 2022-05-09 22:40:34 -07:00
Yakov Olkhovskiy
e04dc446b6 test added 2022-05-10 00:12:36 -04:00
Dmitry Novik
adcb792f0d
Merge pull request #35921 from ClickHouse/enable-memory-overcommit 
Enable memory overcommit
 2022-05-10 02:09:18 +02:00