thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-13 18:02:24 +00:00
Code Issues Packages Projects Releases Wiki Activity
89,093 Commits 9,328 Branches 1,783 Tags 2.1 GiB
7cd7782e4f
Commit Graph

89093 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Dmitry Novik
5128afb0b7 Do not prepend whitespace in grouping sets 2022-05-13 16:14:41 +00:00
Dmitry Novik
fb5766287c Cleanup GROUPING SETS formating 2022-05-13 16:14:41 +00:00
kssenii
5dd3ad08a1 Add S3Requests metric 2022-05-13 18:12:46 +02:00
avogar
e170a1e6b0 Fix build 2022-05-13 15:57:24 +00:00
Dmitry Novik
45585f0e7f Fix possible overflow in OvercommitRatio 2022-05-13 15:39:05 +00:00
Yakov Olkhovskiy
262cc90067 maybe_unused 2022-05-13 11:31:42 -04:00
Alexander Gololobov
fe2aa1861f
Merge pull request #37033 from qieqieplus/array-distance-functions 
Distance/ Norm functions for array
 2022-05-13 17:28:19 +02:00
kssenii
570d006655 Merge 2022-05-13 17:20:10 +02:00
kssenii
d8ec49d9b3 Fix 2022-05-13 17:17:13 +02:00
Yakov Olkhovskiy
2e0aaa0f89 #if instead of #ifdef 2022-05-13 11:03:13 -04:00
Dmitry Novik
ae81268d4d Try to compute helper column lazy 2022-05-13 14:55:50 +00:00
Yakov Olkhovskiy
45bb6ed119 include <Common/config.h> to header 2022-05-13 10:48:51 -04:00
avogar
53d0bb46d6 Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-external-tables-in-hedged-connections 2022-05-13 14:37:58 +00:00
kssenii
0870b9376c Fix 2022-05-13 16:30:59 +02:00
Yakov Olkhovskiy
7f33ac22f4 include config_core.h 2022-05-13 10:30:42 -04:00
Kseniia Sumarokova
3061c557cc
Merge pull request #37182 from Vxider/wv-watch-bug 
Fix bug for WATCH EVENTS query in WindowView
 2022-05-13 16:15:32 +02:00
Mikhail Artemenko
c14fae24c4 run black for python files 2022-05-13 16:55:42 +03:00
Nikolai Kochetov
b2c363a38a
Merge pull request #37157 from ClickHouse/try-enable-cache-for-https-session 
Enable DNS cache for HTTPSClientSession.
 2022-05-13 15:42:30 +02:00
Igor Nikonov
74be0946ff Test fixes 2022-05-13 13:33:45 +00:00
Vxider
273827187c update test 2022-05-13 13:26:38 +00:00
mergify[bot]
603bd85fbe
Merge branch 'master' into function-h3-k-ring-add-cast 2022-05-13 13:26:09 +00:00
Maksim Kita
ef7e21ea46 Function h3kRing added cast 2022-05-13 15:20:04 +02:00
Yakov Olkhovskiy
da624ab297 include Common/config.h 2022-05-13 09:12:23 -04:00
Igor Nikonov
8d5cabc8f7 Fix style warnings and review comments 2022-05-13 11:27:17 +00:00
Kseniia Sumarokova
0f07c41f19
Update MergeTreeIOSettings.h 2022-05-13 13:26:47 +02:00
kssenii
5b9b053c61 Fix encrypted 2022-05-13 13:22:29 +02:00
Kruglov Pavel
c8481b018d
Merge pull request #37156 from azat/reload-listen-directives 
Reload listen_reuse_port/listen_backlog from config
 2022-05-13 13:22:12 +02:00
vdimir
976a274f91
Add comment for visiting children of select in ApplyWithGlobalVisitor 2022-05-13 10:53:26 +00:00
vdimir
dc2a90a344
ApplyWithGlobalVisitor for WITH in subquery 2022-05-13 10:40:51 +00:00
Kseniia Sumarokova
6d5137879b
Merge pull request #37054 from kssenii/better-read-from-cache 
Better read from cache buffer
 2022-05-13 12:20:06 +02:00
Alexander Tokmakov
432f1d570c
Merge pull request #37153 from azat/fix-optimize_skip_unused_shards_rewrite_in 
Fix implicit cast for optimize_skip_unused_shards_rewrite_in
 2022-05-13 12:58:28 +03:00
Vxider
07aaa92299 fix windowview watch 2022-05-13 09:25:50 +00:00
Igor Nikonov
f423e9bf40 Ignore AUTO_INCREMENT in CREATE TABLE to simplify migration from MySQL 
Issue #35893
 2022-05-13 09:22:52 +00:00
qieqieplus
8b3fb22c6d check array sizes for short cut 2022-05-13 17:05:18 +08:00
Vitaly Baranov
dfa1053b9f Use query scopes for async backup/restore. 2022-05-13 10:35:02 +02:00
Azat Khuzhin
b0c72b6104 tests/integration: add prefix match for skipped tests 
This way you can specify only file/module, or test name without
parameters.

Since, at least one, test that we care about
(test_distributed_respect_user_timeouts/test.py::test_reconnect) was not
runned sequentially [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/37138/d7f5db4143c559bb8044058a653956945897e724/integration_tests__asan__actions__[2/3].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-13 11:19:11 +03:00
Antonio Andelic
e4cd605edb
Merge pull request #37100 from ucasfl/fix-s3 
fix wrong argument in proxy resolver of DiskS3
 2022-05-13 10:00:13 +02:00
Azat Khuzhin
f886c4237e tests/integration: fix possible race for iptables user rules inside containers 
It is possible for network PartitionManager to work incorrectly, because
of how docker setting up forward to DOCKER-USER chain, it first removes
forward and then adds it back (see [1] and [2]), however this introduce
race for a short period of time, and this is enough for TCP to
retransmit packets, and breaks network PartitionManager.

  [1]: b1e30e8328/libnetwork/iptables/iptables.go (L638)
  [2]: b1e30e8328/libnetwork/firewall_linux.go (L42)

Here are some details from logs for [3]:

    2022-04-27 03:01:00 [ 621 ] DEBUG : Executing query SELECT node FROM distributed_table ORDER BY node on node2 (cluster.py:2879, query_and_get_error)

  [3]: https://s3.amazonaws.com/clickhouse-test-reports/36295/314d553ab14d30df7508814513506ec09c7c7061/integration_tests__asan__actions__[2/3]/integration_run_parallel1_0.log

This query fails, from the server logs:

    2022.04.27 03:01:00.213101 [ 10 ] {19b1719f-8c39-4e3e-b782-aa4c933650f2} <Debug> executeQuery: (from 172.16.5.1:59008) SELECT node FROM distributed_table ORDER BY node
    ...
    2022.04.27 03:01:03.578439 [ 223 ] {19b1719f-8c39-4e3e-b782-aa4c933650f2} <Debug> Connection (node1:9000): Sent data for 2 scalars, total 2 rows in 0.000284672 sec., 6993 rows/sec., 68.00 B (232.15 KiB/sec.), compressed 0.4594594594594595 times to 148.00 B (505.16 KiB/sec.)
    2022.04.27 03:01:03.590637 [ 223 ] {19b1719f-8c39-4e3e-b782-aa4c933650f2} <Debug> MergingSortedTransform: Merge sorted 3 blocks, 2 rows in 3.371592744 sec., 0.5931914533744174 rows/sec., 94.61 B/sec
    2022.04.27 03:01:03.601256 [ 10 ] {19b1719f-8c39-4e3e-b782-aa4c933650f2} <Information> executeQuery: Read 2 rows, 28.00 B in 3.387950542 sec., 0 rows/sec., 8.26 B/sec.
    2022.04.27 03:01:03.601894 [ 10 ] {19b1719f-8c39-4e3e-b782-aa4c933650f2} <Debug> MemoryTracker: Peak memory usage (for query): 334.38 KiB.

And from docker daemon log:

    time="2022-04-27T03:00:59.916693113Z" level=debug msg="form data: {\"AttachStderr\":true,\"AttachStdin\":false,\"AttachStdout\":true,\"Cmd\":[\"iptables\",\"--wait\",\"-I\",\"DOCKER-USER\",\"1\",\"-p\",\"tcp\",\"-s\",\"172.16.5.2\",\"-d\",\"172.16.5.3\",\"-j\",\"DROP\"],\"Container\":\"b75f3b68cda51386bfbb9cceb67e92c4d217a5a1660bde2470b583cb1f4c7fc4\",\"Privileged\":true,\"Tty\":false,\"User\":\"\"}"
    time="2022-04-27T03:01:00.030654116Z" level=debug msg="form data: {\"AttachStderr\":true,\"AttachStdin\":false,\"AttachStdout\":true,\"Cmd\":[\"iptables\",\"--wait\",\"-I\",\"DOCKER-USER\",\"1\",\"-p\",\"tcp\",\"-s\",\"172.16.5.3\",\"-d\",\"172.16.5.2\",\"-j\",\"DROP\"],\"Container\":\"b75f3b68cda51386bfbb9cceb67e92c4d217a5a1660bde2470b583cb1f4c7fc4\",\"Privileged\":true,\"Tty\":false,\"User\":\"\"}"
    ...
    time="2022-04-27T03:01:03.515813984Z" level=debug msg="/usr/sbin/iptables, [--wait -t filter -n -L DOCKER-USER]"
    time="2022-04-27T03:01:03.531106486Z" level=debug msg="/usr/sbin/iptables, [--wait -t filter -C DOCKER-USER -j RETURN]"
    time="2022-04-27T03:01:03.535442346Z" level=debug msg="/usr/sbin/iptables, [--wait -t filter -C FORWARD -j DOCKER-USER]"
    time="2022-04-27T03:01:03.555856911Z" level=debug msg="/usr/sbin/iptables, [--wait -D FORWARD -j DOCKER-USER]"
    time="2022-04-27T03:01:03.564905764Z" level=debug msg="/usr/sbin/iptables, [--wait -I FORWARD -j DOCKER-USER]"
    ...
    time="2022-04-27T03:01:03.706374466Z" level=debug msg="form data: {\"AttachStderr\":true,\"AttachStdin\":false,\"AttachStdout\":true,\"Cmd\":[\"iptables\",\"--wait\",\"-D\",\"DOCKER-USER\",\"-p\",\"tcp\",\"-s\",\"172.16.5.3\",\"-d\",\"172.16.5.2\",\"-j\",\"DROP\"],\"Container\":\"b75f3b68cda51386bfbb9cceb67e92c4d217a5a1660bde2470b583cb1f4c7fc4\",\"Privileged\":true,\"Tty\":false,\"User\":\"\"}"
    time="2022-04-27T03:01:03.968077970Z" level=debug msg="form data: {\"AttachStderr\":true,\"AttachStdin\":false,\"AttachStdout\":true,\"Cmd\":[\"iptables\",\"--wait\",\"-D\",\"DOCKER-USER\",\"-p\",\"tcp\",\"-s\",\"172.16.5.2\",\"-d\",\"172.16.5.3\",\"-j\",\"DROP\"],\"Container\":\"b75f3b68cda51386bfbb9cceb67e92c4d217a5a1660bde2470b583cb1f4c7fc4\",\"Privileged\":true,\"Tty\":false,\"User\":\"\"}"

I've tried multiple ways of fixing this:

- Creating separate chain for rules from PartitionManager (DOCKER-USER-CLICKHOUSE)
  But it is created only once, and docker places new rules on top of the
  FORWARD chain, so it will not work, since it will not receive any
  packets

- Use DOCKER-USER, but replace iptables with a wrapper ([script]), that
  will ignore recreating of a rule for forward to DOCKER-USER, but this
  will not work too, since new docker rules will be created on top of
  FORWARD chain, and so DOCKER-USER will packets.

  [script]:

    if [[ "$*" =~ "-D FORWARD -j DOCKER-USER" ]]; then
        exit 0
    fi
    if [[ "$*" =~ "-I FORWARD -j DOCKER-USER" ]]; then
        if iptables.real iptables -C FORWARD -j DOCKER-USER; then
            exit 0
        fi
    fi

- And the only way to avoid flakiness for this case, is to forbid
  parallel execution for tests with PartitionManager.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-13 10:57:24 +03:00
Azat Khuzhin
9899b3f92d tests/integration: remove superfluous import of PartitionManager 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-13 10:57:24 +03:00
Azat Khuzhin
0fb9553e1c tests/integration: use no-resolve and verbose for iptables --list 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-13 10:57:24 +03:00
Azat Khuzhin
5c1b0f3a76 tests/integration: add logging for NetworkManager 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-13 10:57:24 +03:00
mergify[bot]
2fdd305ef1
Merge branch 'master' into array-distance-functions 2022-05-13 07:56:57 +00:00
Rich Raposa
2edc03b3d4
Merge pull request #37170 from DanRoscigno/reorder-example-datasets 
Docs: move UK Property and Taxi datasets up
 2022-05-12 16:51:22 -06:00
Yakov Olkhovskiy
323cc3597a fix ifdef 2022-05-12 18:37:52 -04:00
Yakov Olkhovskiy
a228d7ac0e variables naming fixed 2022-05-12 18:10:19 -04:00
kssenii
cde5cecc4d One more 2022-05-12 23:57:25 +02:00
Dan Roscigno
8e5a726c87 Empty-Commit 2022-05-12 17:39:29 -04:00
Kseniia Sumarokova
320efb2221
Merge branch 'master' into better-read-from-cache 2022-05-12 23:35:39 +02:00
Kseniia Sumarokova
3fd9517ff8
Merge pull request #37130 from ClickHouse/kssenii-patch-3 
Update default remote fs read method in ReadSettings
 2022-05-12 23:34:28 +02:00
Maksim Kita
2bf6d9c028
Merge pull request #37154 from azat/context-const 
Add const qualifier for few methods from Context
 2022-05-12 22:20:39 +02:00