thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 02:41:59 +00:00
Code Issues Packages Projects Releases Wiki Activity
104,231 Commits 9,328 Branches 1,783 Tags 2.1 GiB
1309e780ca
Commit Graph

1257 Commits

Author SHA1 Message Date
Alexey Milovidov
29e6a7ed8c Keep controls in place when the page is scrolled horizontally 2022-05-24 04:11:32 +02:00
Dmitry Novik
e9187ec4b7 Overcommit: update defaults, exception message and add ProfileEvent 2022-05-23 14:35:09 +00:00
Vitaly Baranov
58f4a86ec7 Rework notifications used in access management. 2022-05-21 10:15:39 +02:00
Roman Vasin
83958d2549 Add more intergration tests. Add documenation text. Change thread_factor to 3. Add logging. Set default value = -1 to see the results CI tests. 2022-05-20 15:52:50 +03:00
Sergei Trifonov
2450ab9079
Merge pull request #36686 from azat/disk-least-used 
Implement least_used load balancing algorithm for disks inside volume
 2022-05-20 09:16:07 +02:00
Robert Schulze
c0d5705891
Merge pull request #37326 from ClickHouse/fix-marks-cache-size-cfg 
Provide default value for mark_cache_size
 2022-05-19 20:20:25 +02:00
Azat Khuzhin
ba26b3cf4c Implement lead_used load balancing algorithm for disks inside volume 
v2: rebase on top removed raid1
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-19 14:10:11 +03:00
Azat Khuzhin
456444eadb Add storage_configuration example into config.xml 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-19 14:10:11 +03:00
Robert Schulze
a2214760c6
Remove mark_cache_size from further files 
- changed config.xml/yaml files used by CH's own internal tests which
  are (hopefully) not sensitive to mark_cache_size being set or not

- further occurrences exist but changing them seems a bad idea (e.g.
  because they are in customer-provided data)
 2022-05-18 22:09:14 +02:00
Robert Schulze
8ca58cd5a5
Provide default value for mark_cache_size 
I played around with my local config.xml file. The minimal working example is this:
<?xml version="1.0"?>
<clickhouse>
    <mark_cache_size>5368709120</mark_cache_size>
    <listen_host>localhost</listen_host>
    <tcp_port>9000</tcp_port>
    <users_config>users.xml</users_config>
    <logger><console>true</console></logger>
</clickhouse>

Not specifying mark_cache_size made the server not start up:
  2022.05.18 12:15:06.549078 [ 8728320 ] {} <Error> Application: Not found: mark_cache_size

Looking at ClickHouse's ca. 100 server configuration options +
sub-options, it seems that mark_cache_size is NOT special enough to
require explicit configuration but instead that the behavior was
unintended because no default value was provided.
 2022-05-18 15:28:19 +02:00
Roman Vasin
e966016c1a Corrected all issues found by Style Check (actions). Renamed setGlobalMaxThreads into setTotalMaxThreads. 2022-05-17 19:35:40 +03:00
Vitaly Baranov
f34a5cdee2
Merge branch 'master' into ON_CLUSTER-grant 2022-05-17 13:21:20 +02:00
Vitaly Baranov
23a61b1596
Merge pull request #36700 from azat/fix-server-reload 
Fix server reload on port change (do not wait for current connections from query context)
 2022-05-17 12:25:21 +02:00
Roman Vasin
7cc63f3eed Removed logging code 2022-05-17 10:02:54 +03:00
Roman Vasin
159ea14739 Code cleanup. Corrected Loggers path. Refactoring Context code. 2022-05-16 19:48:17 +03:00
Alexey Milovidov
434b8fbf14
Revert "Remove height restrictions from the query div in play web tool." 2022-05-16 19:20:30 +03:00
Azat Khuzhin
c8420c0dbe Remove duplication for server connections check 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-16 16:22:58 +03:00
Azat Khuzhin
e503f1f22c Keep reloaded servers and currently active in one list 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-16 16:22:58 +03:00
Azat Khuzhin
f15a5461be Fix server reload on port change (do not wait for current connections) 
If you will execute 'SYSTEM RELOAD CONFIG' via, i.e., TCP protocol, then
reload on port change will endlessly wait for connection from which this
query had been issued, and you will see the following message in the
logs:

    2022.04.28 03:34:57.552513 [ 37101 ] {b41d855c-4dbf-470a-a144-c6ae5a1abda8} <Debug> executeQuery: (from 127.0.0.1:11774) system reload config
    ...
    2022.04.28 03:34:57.710640 [ 37101 ] {b41d855c-4dbf-470a-a144-c6ae5a1abda8} <Information> Application: Stopped listening for http://127.0.0.1:18123
    2022.04.28 03:34:57.798774 [ 37101 ] {b41d855c-4dbf-470a-a144-c6ae5a1abda8} <Information> Application: Stopped listening for native protocol (tcp): 127.0.0.1:19000
    ...
    2022.04.28 03:34:57.901375 [ 37101 ] {b41d855c-4dbf-470a-a144-c6ae5a1abda8} <Debug> Application: Server finished: http://127.0.0.1:18123
    2022.04.28 03:34:57.901455 [ 37101 ] {b41d855c-4dbf-470a-a144-c6ae5a1abda8} <Trace> Application: Waiting server to finish: native protocol (tcp): 127.0.0.1:19000
    2022.04.28 03:34:58.001717 [ 37101 ] {b41d855c-4dbf-470a-a144-c6ae5a1abda8} <Trace> Application: Waiting server to finish: native protocol (tcp): 127.0.0.1:19000
    2022.04.28 03:34:58.101881 [ 37101 ] {b41d855c-4dbf-470a-a144-c6ae5a1abda8} <Trace> Application: Waiting server to finish: native protocol (tcp): 127.0.0.1:19000
    ...
    2022.04.28 03:35:01.707951 [ 37101 ] {b41d855c-4dbf-470a-a144-c6ae5a1abda8} <Trace> Application: Waiting server to finish: native protocol (tcp): 127.0.0.1:19000

But waiting for the current connection will never ends.

So instead of waiting directly from the query context (SYSTEM RELOAD
CONFIG) do this in background (actually not even in background, but
check on server reload and on exit).

v0: just don't wait for the servers
v2: fix use-after-free by removing dependency from server in handlers
v3: wait servers in background to avoid use-after-free of the context
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-16 16:22:58 +03:00
Azat Khuzhin
01e1c5345a Add separate CLUSTER grant 
In case you have different roles for the same user on multiple clusters,
ON CLUSTER query can help to overcome some limitations.

Consider the following example:
- cluster_with_data, dev_user (readonly=2)
- stage_cluster, dev_user (readonly=0)

So when you will execute the following query from stage_cluster, it will
be successfully executed, since ON CLUSTER queries has different system
profile:

    DROP DATABASE default ON CLUSTER cluster_with_data

This is not 100% safe, but at least something.

Note, that right now only ON CLUSTER query it self is supported, but
separate clusters are not (i.e. GRANT CLUSTER some_cluster_name TO
default), since right now grants sticked to database+.

v2: on_cluster_queries_require_cluster_grant
v3: fix test and process flags as bit mask
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-16 13:57:45 +03:00
Robert Schulze
e3cfec5b09
Merge remote-tracking branch 'origin/master' into clangtidies 2022-05-16 10:12:50 +02:00
Roman Vasin
5dcd374c85 Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-445 2022-05-16 11:09:05 +03:00
Roman Vasin
b06d84b3e5 renamed global_max_threads parameter to total_max_threads 2022-05-13 16:56:03 +03:00
Roman Vasin
87f25c6864 Added thread_factor = 2 constant for initial value calculation of global_max_threads 2022-05-13 15:02:45 +03:00
Roman Vasin
d94e9c4263 For value -1 of global_max_threads replaced physical cores count by logical cores count 2022-05-13 10:56:34 +03:00
Azat Khuzhin
5086d8de29 Reload listen_reuse_port/listen_backlog from config 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-12 17:05:39 +03:00
Roman Vasin
925a94a656 Implemented -1 value handling for global_max_threads parameter 2022-05-12 16:56:24 +03:00
Sergei Trifonov
9800d80a29
update sanity checks warning message text 2022-05-10 09:31:22 +02:00
Dmitry Novik
adcb792f0d
Merge pull request #35921 from ClickHouse/enable-memory-overcommit 
Enable memory overcommit
 2022-05-10 02:09:18 +02:00
guykohen
34c494f403 Remove height restrictions from the query div in play web tool. 2022-05-09 13:32:26 -04:00
mergify[bot]
8ea87a3530
Merge branch 'master' into enable-memory-overcommit 2022-05-09 14:00:54 +00:00
Kruglov Pavel
23bd9390a6
Merge pull request #36997 from vitlibar/users-without-assigned-policies-can-view-rows 
Users without assigned row policies can view rows
 2022-05-09 12:04:50 +02:00
Robert Schulze
1b81bb49b4
Enable clang-tidy modernize-deprecated-headers & hicpp-deprecated-headers 
Official docs:

  Some headers from C library were deprecated in C++ and are no longer
  welcome in C++ codebases. Some have no effect in C++. For more details
  refer to the C++ 14 Standard [depr.c.headers] section. This check
  replaces C standard library headers with their C++ alternatives and
  removes redundant ones.
 2022-05-09 08:23:33 +02:00
Vitaly Baranov
69bec2f377 Users without assigned row policies can view rows now. 2022-05-07 14:50:24 +02:00
Alexey Milovidov
b2945766d0
Merge pull request #36988 from ClickHouse/handle-nullable-numbers-in-play-ui 
Correct styling of Nullable numbers in PlayUI
 2022-05-07 13:06:25 +03:00
Alexey Milovidov
a88e9b74a2 Correct styling of Nullable numbers in PlayUI 2022-05-06 21:50:28 +02:00
Alexey Milovidov
8b26c5fe02
Merge pull request #36838 from ClickHouse/config-update 
Disable `merge_tree_metadata_cache` by default
 2022-05-06 01:59:01 +03:00
mergify[bot]
1a92a34514
Merge branch 'master' into enable-memory-overcommit 2022-05-02 17:04:08 +00:00
Alexey Milovidov
0b9f8d81e4 Add cool processing of empty resultsets 2022-05-02 03:48:48 +02:00
Alexey Milovidov
2f0c421d99
Update play.html 2022-05-02 04:24:40 +03:00
Alexey Milovidov
8b366cbb02
Update play.html 2022-05-02 04:23:52 +03:00
Alexey Milovidov
be7a4b8b30 Make it work automatically; unify the code 2022-05-02 03:19:57 +02:00
Alexey Milovidov
474a805ea7 Merge branch 'play-vertical-format' of github.com:vdimir/ClickHouse into vdimir-play 2022-05-02 02:44:09 +02:00
Nikita Mikhaylov
f2d73635ef Disable 2022-05-01 17:14:52 +00:00
Alexey Milovidov
69ae22070f
Merge pull request #36824 from ClickHouse/log-test-sanity-check 
Add sanity check for log level. Fix style.
 2022-05-01 14:03:22 +03:00
Nikita Taranov
92f8244932
Fix certs (finishing pr #36457) (#36747) 2022-05-01 12:59:19 +02:00
Alexey Milovidov
47988db1a5 Add sanity check for log level. Fix style. 2022-04-30 13:19:25 +02:00
Alexey Milovidov
e0b2365945
Merge pull request #36472 from ClickHouse/sanity-checks-message-fix 
more specific warning text about low disk space
 2022-04-30 09:44:21 +03:00
Alexey Milovidov
1ddb04b992
Merge pull request #36715 from amosbird/refactorbase 
Reorganize source files so that base won't depend on Common
 2022-04-30 09:40:58 +03:00
vdimir
ea4e9db3da
play.html: no header in transposed table 2022-04-29 20:05:03 +02:00
vdimir
277afd0c48
play.html: add button to transpose table 2022-04-29 20:01:24 +02:00
Nikolai Kochetov
5807ca47b0
Merge pull request #36425 from ClickHouse/pool-size-hot-reload 
Allow to increase number of threads and tasks for background executors
 2022-04-29 17:24:12 +02:00
Maksim Kita
5b9bb36f80
Merge pull request #36753 from kitaisreal/added-user-defined-path-config-setting 
Added user_defined_path config setting
 2022-04-29 11:10:51 +02:00
Amos Bird
4a5e4274f0
base should not depend on Common 2022-04-29 10:26:35 +08:00
Maksim Kita
04429d85db Added user_defined_path config setting 2022-04-28 22:13:33 +02:00
Alexey Milovidov
770ce07b1a Change Maven URL example for JDBC bridge in ClickHouse config 2022-04-28 05:54:39 +02:00
mergify[bot]
a13012e271
Merge branch 'master' into sanity-checks-message-fix 2022-04-28 02:50:16 +00:00
Alexey Milovidov
39467ec4aa
Update play.html 2022-04-26 08:47:56 +03:00
mergify[bot]
7c32649014
Merge branch 'master' into cleanup-play-ui 2022-04-26 05:47:32 +00:00
mergify[bot]
74aa65e36e
Merge branch 'master' into sanity-checks-message-fix 2022-04-26 05:45:11 +00:00
mergify[bot]
9c3605712b
Merge branch 'master' into sanity-checks-message-fix 2022-04-25 17:37:55 +00:00
mergify[bot]
cc8513fb79
Merge branch 'master' into cleanup-play-ui 2022-04-25 17:15:08 +00:00
Azat Khuzhin
6df54611a9 Log which taskstats provider is in use 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
v2: add missing __builtin_unreachable() for gcc built [1]
    [1]: https://s3.amazonaws.com/clickhouse-builds/36351/99e72a960325cc1fb3cd3d6a1b56cf2b1cc2d733/binary_gcc/build_log.log
(cherry picked from commit 2be022cb56)
 2022-04-25 07:56:54 +03:00
Alexey Milovidov
74bb6349c2
Revert "Strict taskstats parser" 2022-04-25 01:15:08 +03:00
mergify[bot]
989f7c76ab
Merge branch 'master' into cleanup-play-ui 2022-04-24 19:45:18 +00:00
Alexey Milovidov
8af3159916
Merge pull request #36351 from azat/netlink-strict 
Strict taskstats parser
 2022-04-24 05:14:06 +03:00
Alexey Milovidov
a1eddd1be0 Cleanup CSS in Play UI 2022-04-23 20:42:10 +02:00
Roman Vasin
0c467872d9 Moved global_max_threads parameter from users.xml file to config.xml file 2022-04-22 17:17:29 +03:00
Nikita Mikhaylov
aaa7112405 Even better 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
e475849761 Style and Docs update 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
f95a63a7f0 Beautify 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
36bdee0499 Deleted settings 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
31ccb9c1c3 Done 2022-04-22 13:56:16 +00:00
Alexey Milovidov
f31cad52ab Preparation 2022-04-22 03:21:29 +02:00
Sergei Trifonov
cda9160c2f fix build 2022-04-21 07:48:56 +02:00
Sergei Trifonov
69403840e2 add paths 2022-04-21 07:45:08 +02:00
Sergei Trifonov
a2365c5c83 more specific warning text about low disk space 2022-04-21 07:32:34 +02:00
Azat Khuzhin
2be022cb56 Log which taskstats provider is in use 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
v2: add missing __builtin_unreachable() for gcc built [1]
    [1]: https://s3.amazonaws.com/clickhouse-builds/36351/99e72a960325cc1fb3cd3d6a1b56cf2b1cc2d733/binary_gcc/build_log.log
 2022-04-20 21:39:26 +03:00
mergify[bot]
c026dbf51c
Merge branch 'master' into interserver_listen_port 2022-04-19 11:44:01 +00:00
Alexey Milovidov
58fc4e033a Remove trash 2022-04-15 22:54:08 +02:00
Alexey Milovidov
efbb29539f Fix Play UI for some resultsets 2022-04-15 03:13:40 +02:00
mergify[bot]
078d8baf1d
Merge branch 'master' into enable-memory-overcommit 2022-04-11 17:09:02 +00:00
Alexander Tokmakov
7f54e7b422 Merge branch 'master' into mvcc_prototype 2022-04-07 15:14:06 +02:00
tavplubix
6e9c028bbb
Merge pull request #35956 from Algunenano/limit_dns_retries 
DNS cache: Add option to drop elements after several consecutive failures
 2022-04-07 13:20:36 +03:00
Nikita Mikhaylov
d7071e8596 Style 2022-04-05 17:04:53 +02:00
mergify[bot]
bc882ded42
Merge branch 'master' into enable-memory-overcommit 2022-04-05 14:46:18 +00:00
Raúl Marín
8a05cf3927 DNS cache: Add option to drop elements after several consecutive failures 2022-04-05 13:00:27 +02:00
Dmitry Novik
b4fed414e3 Enable memory overcommit 2022-04-04 14:28:05 +00:00
Alexander Tokmakov
a2167f12b8 Merge branch 'master' into mvcc_prototype 2022-04-04 14:24:23 +02:00
Azat Khuzhin
14538f6456 Add system.processors_profile_log 
This is the system table that will contain Processors level profiling.

v2: one entry per Processor, not 3 (PortFull/NeedData/work())
v3: us over ms
v4: Enable processors_profile_log table by default

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 13:19:41 +03:00
Alexey Milovidov
808c802cae Disable session_log 2022-04-03 05:07:37 +02:00
Alexey Milovidov
17107cb1dc Update examples and remove obsolete embedded dictionaries from configs 2022-04-03 01:21:48 +02:00
Nir Peled
409eccec84 When showing the animation, set display to inline-block instead of inline, otherwise the animation won't start 2022-04-01 19:29:13 -04:00
Nir Peled
4810936eaa Added an animation to the play ui hourglass when a query is running 2022-04-01 19:24:14 -04:00
mergify[bot]
3183b61c74
Merge branch 'master' into mvcc_prototype 2022-03-31 12:27:09 +00:00
alesapin
ae3bcbb99b Fix logs space check 2022-03-31 00:22:36 +02:00
alesapin
c6c0c91b39 Followup 2022-03-29 18:20:20 +02:00
Alexander Tokmakov
287d858fda Merge branch 'master' into mvcc_prototype 2022-03-29 16:24:12 +02:00
alesapin
4db85e2809 Fix build 2022-03-29 14:20:46 +02:00
alesapin
ed2c461fe1 Merge branch 'master' into startup-sanity-checks 2022-03-29 14:09:20 +02:00
taiyang-li
38702be40c Merge remote-tracking branch 'ck/master' into rocksdb_metacache 2022-03-28 21:03:22 +08:00
alesapin
d3fbd9b2ad Fix style 2022-03-24 11:25:28 +01:00
alesapin
93d1b1d198 Merge branch 'master' into startup-sanity-checks 2022-03-24 11:17:06 +01:00
Alexander Tokmakov
bcec53aadc Merge branch 'master' into mvcc_prototype 2022-03-23 11:28:43 +01:00
Antonio Andelic
9829f2d051 Merge branch 'master' into parallel-downloading-url-engine 2022-03-23 07:41:14 +00:00
taiyang-li
8dbf1c60e7 merge master and fix conflict 2022-03-23 11:36:50 +08:00
alesapin
70d410dc4d Fix keeper client timeout bug 2022-03-22 11:41:50 +01:00
Antonio Andelic
103a3fa140 Merge branch 'master' into parallel-downloading-url-engine 2022-03-17 10:11:58 +00:00
Alexander Tokmakov
4b3e13a4fe Merge branch 'master' into mvcc_prototype 2022-03-16 21:06:19 +01:00
Vitaly Baranov
39614e6e15
Merge pull request #35276 from vitlibar/fix-code-style-and-minor-corrections-after-allow-no-password 
Fix code style and other minor corrections after implementing allow_no_password.
 2022-03-16 18:55:05 +01:00
Antonio Andelic
0979155f2f Address PR comments 2022-03-16 15:45:17 +00:00
alexander goryanets
ee29e93fc0
use interserver_listen_host param for configure interserver listen_host 2022-03-16 12:22:05 +00:00
alexander goryanets
22a91d190e
introduce interserver_listen_host param 2022-03-16 12:22:03 +00:00
Antonio Andelic
2291b7c1e1 Merge branch 'master' into parallel-downloading-url-engine 2022-03-16 09:59:52 +00:00
Nikolai Kochetov
a380aa6b8a
Merge pull request #35294 from ClickHouse/reload-remote_url_allow_hosts 
Reload remote_url_allow_hosts after config update.
 2022-03-15 22:07:16 +01:00
Alexander Tokmakov
1f571b7734 Merge branch 'master' into mvcc_prototype 2022-03-15 14:45:06 +01:00
Nikolai Kochetov
97aa6c82ce Reload remote_url_allow_hosts after config update. 2022-03-15 13:00:31 +00:00
Vladimir C
af88d3f133
Merge pull request #35173 from bigo-sg/dump_ast_in_dot 2022-03-15 11:18:41 +01:00
Antonio Andelic
067b79b00b Merge branch 'master' into parallel-downloading-url-engine 2022-03-15 07:55:41 +00:00
Alexander Tokmakov
9702b5177d Merge branch 'master' into mvcc_prototype 2022-03-14 21:45:38 +01:00
Alexander Tokmakov
278d779a01 log cleanup, more comments 2022-03-14 21:43:34 +01:00
Vitaly Baranov
1eb2e8693e Fix code style and other minor corrections after implementing allow_no_password. 2022-03-14 20:55:34 +01:00
Vitaly Baranov
4af61fb9d3
Merge pull request #34738 from DevTeamBK/Issue-33953 
New setting in Server Configuration to on/off AuthType Plaintext_password and No_password
 2022-03-14 17:09:46 +01:00
Antonio Andelic
d3353f3f0c Merge branch 'master' into parallel-downloading-url-engine 2022-03-14 12:47:23 +00:00
alesapin
98144bc1f5
Merge pull request #35031 from ClickHouse/standalone_keeper_build 
Building small keeper binary
 2022-03-14 11:27:25 +01:00
Alexey Milovidov
8420b5c29a Change timezone example in server config 2022-03-11 23:33:26 +01:00
alesapin
33ff2f76e5 Merge branch 'master' into standalone_keeper_build 2022-03-11 13:58:44 +01:00
Antonio Andelic
28e9508c4e Improve shared pool and add settings 2022-03-11 08:33:34 +00:00
HeenaBansal2009
3f031df225 Code refactoring 2022-03-10 22:22:51 -08:00
taiyang-li
2ae5129e2d modify play.tml 2022-03-11 12:13:46 +08:00
HeenaBansal2009
3ce9397246 Added Suggestions from Code review 2022-03-09 20:35:01 -08:00
taiyang-li
b4174b0bef merge master and fix conflicts 2022-03-08 11:39:25 +08:00
alexey-milovidov
ee619dc68e
Merge pull request #35068 from ClickHouse/PlayUI_theme_fixes 
Added 'theme' GET parameter to PlayUI + theme selection according to OS fixes
 2022-03-05 02:59:27 +03:00
Nir Peled
9c024c9617 Changed the Play UI to select a theme by the following priority: 
* 'theme' GET parameter
* 'theme' in localStorage
* According to OS preference (didn't work before)

Also, theme is only saved in localStorage if a user specifically clicked on one theme icon or the other.
 2022-03-04 18:44:45 -05:00
alexey-milovidov
7da2c5f927
Update play.html 2022-03-05 02:04:49 +03:00
alesapin
5416b567d5 Merge branch 'master' into standalone_keeper_build 2022-03-04 13:35:07 +01:00
alesapin
0eb7d28192 Building small keeper binary 2022-03-03 21:27:46 +01:00
Vladimir C
eb68c21152
Merge pull request #34995 from vdimir/fix-aarch64 2022-03-03 12:18:28 +01:00
Azat Khuzhin
798da0c314 Introduce safeExit() helper (_exit() compatible with TSan) 
v2: add missing defines.h header
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-02 22:17:17 +03:00
vdimir
fc5e941f79
Add option to clickhouse-test to skip aarch64 build 2022-03-02 14:18:24 +00:00
HeenaBansal2009
aa8494a808 Fix: System Reload Config Failure 2022-02-28 10:51:49 -08:00
HeenaBansal2009
1b263f0c15 Added FT testcase 2022-02-18 12:58:46 -08:00
HeenaBansal2009
d16cae53b4 Initial Commit for Plaintext password feature 2022-02-17 21:25:18 -08:00
Sergei Trifonov
0f66743104 support determine available memory for FreeBSD 2022-02-15 13:59:00 +03:00
serxa
7ea3c003d5 fix style 2022-02-15 10:39:45 +00:00
Sergei Trifonov
a819cb7afe avoid unhandled exceptions on sanity checks at server startup 2022-02-15 12:56:21 +03:00
李扬
f52b67b939
Merge branch 'master' into rocksdb_metacache 2022-02-15 02:16:29 -06:00
alexey-milovidov
bc206ec423
Merge pull request #31182 from ClickHouse/memory-overcommit 
Memory overcommit
 2022-02-14 23:55:02 +03:00
Sergei Trifonov
3d66acc56b
Apply suggestions from code review 
Co-authored-by: alesapin <alesapin@clickhouse.com>
 2022-02-14 17:16:05 +03:00
Sergei Trifonov
2b0b03276b add simple sanity checks for server startup 2022-02-13 19:58:38 +03:00
李扬
daa27d0bda
Merge branch 'master' into rocksdb_metacache 2022-02-12 07:50:12 -06:00
mergify[bot]
cb3e5f8538
Merge branch 'master' into memory-overcommit 2022-02-10 11:01:43 +00:00
Dmitry Novik
d2e1f66c57 Fix test 2022-02-10 01:01:14 +00:00
Vitaly Baranov
1341b4b4de Rename QueryInfo's field result_compression -> transport_compression_type and change its type for better consistency. 
Make the previous field obsolete.
 2022-02-10 02:46:05 +07:00
taiyang-li
b6132d490f merge master and solve conflict 2022-02-08 15:24:59 +08:00
alexey-milovidov
665fdc9187
Merge pull request #31257 from FArthur-cmd/merge_15765 
Merge #15765
 2022-02-04 00:39:57 +03:00
FArthur-cmd
73f579d71c Comment dhParamsFile. It should be uncommented only if it has correct path 2022-02-03 13:31:52 +00:00
FArthur-cmd
f327e67232 correct config 2022-02-02 20:01:25 +00:00
Amos Bird
ec7d367814
DiskLocal checker 
Add DiskLocal checker so that ReplicatedMergeTree can recover data when some of its disks are broken.
 2022-02-01 05:55:27 +08:00
FArthur-cmd
f20f2f6beb fix test 2022-01-31 13:39:39 +00:00
FArthur-cmd
7811f5d599 try uncomment config 2022-01-31 11:18:57 +00:00
FArthur-cmd
cb4ad97d45 add test 2022-01-28 17:37:52 +00:00
taiyang-li
3de8bde7ce Merge remote-tracking branch 'origin/master' into rocksdb_metacache 2022-01-28 09:58:52 +08:00
Filatenkov Artur
fcac77b918
Merge branch 'ClickHouse:master' into merge_15765 2022-01-25 23:21:24 +03:00
alesapin
b8c6481d4c Fix clickhouse server start when replicated access storage depend on keeper 2022-01-25 16:40:12 +03:00
Filatenkov Artur
82b0fc0bd0
Merge branch 'master' into merge_15765 2022-01-24 13:38:53 +03:00
taiyang-li
73def8b483 merge master and solve conflict 2022-01-24 11:01:43 +08:00
Azat Khuzhin
a6d482d3e1 Cleanup jemalloc contrib 2022-01-20 10:02:03 +03:00
Azat Khuzhin
a773e7ff01 Remove unbundled libpqxx support 2022-01-20 10:01:59 +03:00
Azat Khuzhin
126aa4b65c Remove unbundled base64 support 2022-01-20 10:01:58 +03:00
Dmitry Novik
c0970b75ee
Merge branch 'master' into memory-overcommit 2022-01-18 15:30:24 +03:00
Dmitry Novik
83c663e2d6 Cleanup after code review 2022-01-18 12:21:59 +00:00
taiyang-li
3ed13e7896 refactor stateless test and add move part/partition integration test 2022-01-07 21:06:10 +08:00
taiyang-li
3803cc3d5e fix bug 2022-01-06 15:42:12 +08:00
taiyang-li
8e675e9e0a fix building 2022-01-04 15:06:19 +08:00
taiyang-li
94d1f7ccb1 enable continue if cache data corrupted 2022-01-04 12:36:04 +08:00
taiyang-li
e13e1f5d7e add unit test && use enum_name 2022-01-04 11:44:10 +08:00
taiyang-li
3f6d830536 Merge remote-tracking branch 'origin/master' into rocksdb_metacache 2022-01-04 10:00:34 +08:00
taiyang-li
1e102bc1b2 merge master 2022-01-01 09:01:06 +08:00
Mikhail f. Shiryaev
272ea7fc5b
Merge pull request #32609 from cmsxbc/query-kind-concurent_restriction 
add settings: max_concurrent_select_queries and max_concurrent_insert_queries
 2021-12-29 15:23:45 +01:00
mergify[bot]
5070784282
Merge branch 'master' into memory-overcommit 2021-12-29 10:35:32 +00:00
taiyang-li
98f37afc80 fix building 2021-12-28 19:29:01 +08:00
taiyang-li
aa97634d9b merge master 2021-12-28 18:38:50 +08:00
taiyang-li
2a1fe52b9f rename symbols && fix uts && add setting use_metadata_cache 2021-12-28 18:06:13 +08:00
taiyang-li
ae2078a58e Merge branch 'master' into rocksdb_metacache 2021-12-28 15:56:10 +08:00
Vladimir C
bb6fc853e3
Merge pull request #31442 from vdimir/cluster-discovery 2021-12-28 10:53:30 +03:00
李扬
91334ecf50
Merge branch 'master' into rocksdb_metacache 2021-12-27 23:20:41 -06:00
taiyang-li
7dab7caa9d wrap cache related code with USE_ROCKSDB 2021-12-28 11:57:43 +08:00
alesapin
2f9d5af14e Fix clang tidy 2021-12-27 19:54:14 +03:00
vdimir
fbdb5c60bd
initial initialization of cluster discovery in main thread 2021-12-27 15:26:10 +03:00
vdimir
7404205f37
fix cluster discovery startup race 2021-12-27 15:26:09 +03:00
vdimir
e7d3dbeebd
cluster discovery init only with zookeeper 2021-12-27 15:26:08 +03:00
vdimir
94bb7cba62
cluster discovery wip 2021-12-27 15:26:05 +03:00
Vitaly Baranov
f80a3e3247
Merge pull request #30549 from aiven/kmichel-server-reload 
Start/stop servers when `listen_host`/`*_port` changes
 2021-12-27 14:51:00 +03:00
taiyang-li
e7401d2a5e wrap rocksdb metacache related code with USE_ROCKSDB 2021-12-27 11:50:59 +08:00
Kevin Michel
ffc1fca296
Start/stop servers when listen_host/*_port changes 
This allows starting and stopping separately each protocol server
without restarting ClickHouse.

This also allows adding or removing `listen_host` entries, which
start and stops servers for all enabled ports.

When stopping a server, the listening socket is immediately closed
(and available for another server).

Protocols with persistent connections try to wait for any currently
running query to finish before closing the connection, but idle
connection are closed quickly (depending on how often the protocol
is polled).

An extra ProfileEvent is added, `MainConfigLoads`, it is
incremented every time the configuration is reloaded. This helps
when trying to assess whether the new configuration was applied.
 2021-12-24 08:26:02 +01:00
mergify[bot]
d4ec469ed1
Merge branch 'master' into query-kind-concurent_restriction 2021-12-24 07:02:00 +00:00
lgbo-ustc
ccc39bad80 move IRemoteFileMetadata info Storages/Cache 2021-12-23 15:56:33 +08:00
lgbo-ustc
d205f9ecab codes refactor 2021-12-23 11:50:26 +08:00
taiyang-li
2597925724 merge master 2021-12-21 15:55:39 +08:00
lgbo-ustc
71353ceede refactor metadata class factory 2021-12-20 20:33:41 +08:00
liyang
37ba8004ff Speep up mergetree starting up process 2021-12-18 16:39:59 +08:00
alesapin
6bd7e425c6
Merge pull request #22535 from CurtizJ/sparse-serialization 
Sparse serialization and ColumnSparse
 2021-12-17 15:26:17 +03:00
taiyang-li
d033fc4c24 merge master and fix conflict 2021-12-17 15:11:21 +08:00
Anton Popov
16312e7e4a Merge remote-tracking branch 'upstream/master' into HEAD 2021-12-14 18:58:17 +03:00
taiyang-li
ca3f7425a4 fix code 2021-12-14 17:37:31 +08:00
cmsxbc
b30e250eed add max_concurrent_select_queries and max_concurrent_insert_queries 2021-12-14 07:37:38 +00:00
Azat Khuzhin
15bc4c0ca8 Add max_threads_for_connection_collector example and description into config 2021-12-14 10:18:57 +03:00
mergify[bot]
b81d600c1e
Merge branch 'master' into memory-overcommit 2021-12-12 23:01:56 +00:00
李扬
8675086104
Merge branch 'master' into hive_table 2021-12-12 09:01:46 -06:00
Azat Khuzhin
57c027be0e Remove arcadia build support 2021-12-11 21:25:23 +03:00
taiyang-li
02b877c978 move remote buffer from src/IO to src/Storages 2021-12-10 10:27:53 +08:00
Nikita Mikhaylov
dbf5091016
Parallel reading from replicas (#29279) 2021-12-09 13:39:28 +03:00
lgbo-ustc
5a8c56bd55 fixed erros on creating dir on startup 2021-12-07 16:35:08 +08:00
lgbo-ustc
b6c230606e fixed config error in server startup 2021-12-07 14:46:17 +08:00
mergify[bot]
600dcb749a
Merge branch 'master' into memory-overcommit 2021-12-07 00:40:20 +00:00
Anton Popov
54f51444c0 Merge remote-tracking branch 'upstream/master' into HEAD 2021-12-01 15:49:02 +03:00
lgbo-ustc
aaa5d8f002 rewrite thread pool in remote file cache 2021-12-01 10:45:25 +08:00
taiyang-li
c7a0fe467f fix code stylke 2021-11-29 16:22:43 +08:00
taiyang-li
72f60cceb9
Merge branch 'master' into hive_table 2021-11-25 17:33:26 +08:00
Timur Magomedov
08fc832038 Remove OpenCL completely 2021-11-25 00:45:35 +03:00
lgbo
996d7125c0
Merge branch 'master' into hive_table 2021-11-23 10:19:02 +08:00
alesapin
35de9e42d9
Merge branch 'master' into add_4_letter_words_commands 2021-11-22 18:02:02 +03:00
tavplubix
dc6d48ff83
Merge pull request #31292 from stigsb/materialized-mysql-atomic-only 
Drop support for DatabaseOrdinary in MaterializedMySQL
 2021-11-22 14:03:50 +03:00
Alexey Milovidov
334535a4c4 Better exception message when users.xml cannot be loaded due to bad password hash 2021-11-20 10:06:20 +03:00
taiyang-li
e8644807fe merge master and solve conflict 2021-11-19 15:01:58 +08:00
Stig Bakken
ff46e8bb51 Drop support for DatabaseOrdinary in MaterializedMySQL 
1. Dropped support for DatabaseOrdinary for MaterializeMySQL. It
   is marked as experimental, and dropping support makes the code
   more maintaible, and speeds up integration tests by 50%.

2. Get rid of thread name logic for StorageMaterializeMySQL wrapping,
   use setInternalQuery instead (similar to MaterializedPostgreSQL).
 2021-11-18 11:46:51 +01:00
alesapin
5f3eca4cd1 Merge branch 'master' into JackyWoo_add_4_letter_words_commands 2021-11-17 12:59:20 +03:00
lgbo-ustc
a41a917b51 fixed : make recovering local cache meta informations run in backgroup 2021-11-15 16:47:12 +08:00
Tomáš Hromada
445b0ba7cc
Expose all GlobalThreadPool config params (#31285) 2021-11-12 16:24:47 +03:00
lgbo-ustc
9ad441c56d rewrite codes base on comment 2021-11-11 19:50:08 +08:00
taiyang-li
af78116c3f
Merge branch 'master' into hive_table 2021-11-11 11:52:28 +08:00
Artur
4aaa034c8e Correct behaviour if certificate or key was not set 2021-11-10 16:56:07 +00:00
Artur Filatenkov
744d50f328 merge with master 2021-11-10 18:35:35 +03:00
Vitaly Baranov
1579bcfd20
Merge pull request #31178 from vitlibar/backup-engines-and-improvements 
Backup engines and improvements
 2021-11-10 18:20:41 +03:00
taiyang-li
26988ab757
Merge branch 'master' into hive_table 2021-11-10 11:01:03 +08:00
Vitaly Baranov
cf77c0b3fc New syntax for BACKUP/RESTORE: set backup engine explicitly. 2021-11-09 23:16:40 +03:00
Dmitry Novik
9464dc2fd3 Add waiting timeout 2021-11-09 16:02:36 +03:00
Dmitry Novik
0816dedaea Introduce memory overcommit 2021-11-09 16:02:36 +03:00
Anton Popov
13b35a6817 Merge remote-tracking branch 'upstream/master' into HEAD 2021-11-08 21:29:16 +03:00
Anton Popov
66973a2a28 Merge remote-tracking branch 'upstream/master' into HEAD 2021-11-08 21:27:45 +03:00
Nikolai Kochetov
14a3988f8c
Merge pull request #31033 from azat/remap-log 
Log size of remapped memory (remap_executable)
 2021-11-08 17:12:04 +03:00
Artur Filatenkov
d8510583d0 resolve conflict 2021-11-05 17:07:29 +03:00
taiyang-li
36ca0b296b implement hive table engine 2021-11-05 19:55:30 +08:00
alexey-milovidov
74cac98739
Merge pull request #31030 from azat/tests-log-tables 
Remove metric_log/part_log overrides in tests (enabled by default)
 2021-11-04 00:30:43 +03:00
Azat Khuzhin
7769d75088 Log size of remapped memory (remap_executable) 2021-11-04 00:19:00 +03:00
Vitaly Baranov
1f217aeb5a
Merge pull request #30998 from vitlibar/rename-access-control-manager 
Rename AccessControlManager
 2021-11-03 12:20:17 +03:00
Azat Khuzhin
0047d2eb4f Remove metric_log/part_log overrides in tests (enabled by default) 2021-11-03 08:39:36 +03:00
Yuriy Chernyshov
14e7ea6685 Remove remaining usages of Y_IGNORE 2021-11-02 14:41:30 +03:00
Vitaly Baranov
afe2c9c040 Rename AccessControlManager -> AccessControl. 2021-11-02 14:06:20 +03:00
Anton Popov
d50137013c Merge remote-tracking branch 'upstream/master' into HEAD 2021-11-01 16:55:53 +03:00
mergify[bot]
65df198e9b
Merge branch 'master' into conf-clickhouse 2021-10-28 22:19:42 +00:00
JackyWoo
a60663e33d add 4lw commands to keeper 2021-10-27 20:26:42 +08:00
Maksim Kita
b58f819789
Merge pull request #30667 from kitaisreal/compiled-expression-cache-limit-elements-size 
CompiledExpressionCache limit elements size
 2021-10-26 16:43:49 +03:00
Azat Khuzhin
e60c1f1061 programs: switch left configs from <yandex> to <clickhouse> 2021-10-26 08:52:34 +03:00
Maksim Kita
3810baf44c CompiledExpressionCache limit elements size 2021-10-26 00:00:57 +03:00
Vitaly
8a55165c3b
Typo fix in play.html 2021-10-25 11:15:25 +03:00
alesapin
490ca93162
Merge pull request #30372 from ClickHouse/update_keeper_config 
Updatable keeper configuration.
 2021-10-25 10:37:03 +03:00
alexey-milovidov
b0f05f82b9
Merge pull request #30511 from orloffv/patch-3 
Add bytes to stats and human readable rows to play.html
 2021-10-25 09:38:32 +03:00
mergify[bot]
b043a04767
Merge branch 'master' into update_keeper_config 2021-10-22 07:12:42 +00:00
Vitaly
9a64557eba
Update play.html 2021-10-21 19:40:50 +03:00
Vitaly
88bf5d631e
Add bytes to stats and human readable rows 2021-10-21 19:30:52 +03:00
Vitaly
0ea2a37494
Add placeholder to play.html inputs 2021-10-21 18:49:20 +03:00
mergify[bot]
35b5cf23cb
Merge branch 'master' into update_keeper_config 2021-10-20 09:57:09 +00:00
Kseniia Sumarokova
5324cc8359
Merge pull request #30282 from kssenii/fix-local-less-threads 
Less threads in clickhouse-local, fix Ok. printing
 2021-10-19 22:54:28 +03:00
kssenii
e53335bc6f Better way 2021-10-19 08:21:38 +00:00
alesapin
f2d266acce Merge branch 'master' into update_keeper_config 2021-10-19 10:16:57 +03:00
alesapin
bfe2a937eb At least able to start 2021-10-18 18:27:51 +03:00
Anton Popov
d71ffc355a Merge remote-tracking branch 'upstream/master' into HEAD 2021-10-18 15:18:22 +03:00
alesapin
6d24ca4c3a Review fixes 2021-10-18 12:13:24 +03:00
alesapin
f1fe96e194 Merge branch 'master' into debug_keeper 2021-10-18 10:16:07 +03:00
alexey-milovidov
f4bfed9d3a
Merge pull request #29586 from evillique/log_levels_update 
Add log levels updates
 2021-10-17 20:54:46 +03:00
Nikolai Kochetov
067eaadadd Merge branch 'master' into removing-data-streams-folder 2021-10-16 09:46:05 +03:00
Nikolai Kochetov
fd14faeae2 Remove DataStreams folder. 2021-10-15 23:18:20 +03:00
Maksim Kita
7beaeb8b13
Merge branch 'master' into indexcache 2021-10-15 12:59:09 +03:00
Anton Popov
7aa6068fb2 Merge remote-tracking branch 'upstream/master' into HEAD 2021-10-14 19:44:08 +03:00
Nikolay Degterinsky
2da43012b6 Add log levels updates 2021-10-14 16:34:30 +03:00
alesapin
90ff7f05fd Start keeper asynchronously if has connection to other nodes 2021-10-14 13:21:41 +03:00
Maksim Kita
18a980d7b0
Merge pull request #29914 from amosbird/waitunfinish 
Add shutdown_wait_unfinished_queries setting
 2021-10-12 18:42:31 +03:00
Azat Khuzhin
063f9cffab Allow memory profiler under sanitizers 
Only query profiler cannot work reliably with sanitizers (due to
unwinding from signal handler), but memory profiler should be fine.

Plus sometimes the problem appears only on build with sanitizers, so it
will be useful to have memory related profiling in trace_log.

Also there is a flaky check for stateless tests, that uses build with
ASan, and now trace_log there is empty, which sometimes does not allow
to debug further.
 2021-10-11 10:21:26 +03:00
Maksim Kita
85f280f017
Merge pull request #29951 from vitlibar/fix-access-control-shutdown 
Fix shutdown of AccessControlManager
 2021-10-10 23:56:40 +03:00
Vitaly Baranov
28459b282d Fix shutdown of AccessControlManager. Now there cannot be configuration reloading after AccessControlManager has been destroyed. 2021-10-10 13:25:24 +03:00
alexey-milovidov
ee577e1ab4
Merge pull request #29155 from FArthur-cmd/add_CORS 
Add Cors support
 2021-10-09 20:18:17 +03:00
Filatenkov Artur
7753cc9c28
Update users.xml 2021-10-09 16:01:01 +03:00
Filatenkov Artur
98ecd59ebe
Update users.xml 2021-10-09 16:00:37 +03:00
Artur
12f59f2dca refactor 2021-10-09 12:52:20 +00:00
Artur
473f7bee3a refactor 2021-10-09 12:16:37 +00:00
Amos Bird
5cc3793925
Add shutdown_wait_unfinished_queries setting 2021-10-09 11:50:06 +08:00
Azat Khuzhin
71cffbf521 Make memory_profiler_step API cleaner 
Right now to configure memory_profiler_step/total_memory_profiler_step
you need to call:

    MemoryTracker::setOrRaiseProfilerLimit()
    MemoryTracker::setProfilerStep()

But it is easy to forget about setOrRaiseProfilerLimit(), since there is
no even any comments about this.

So instead, make setOrRaiseProfilerLimit() private and call it from
setProfilerStep()
 2021-10-07 10:52:16 +03:00
Azat Khuzhin
74f5b92420 Add listen_backlog to documentation 
Follow-up for: #29643
 2021-10-06 21:09:00 +03:00
Filatenkov Artur
545673248a
Merge branch 'master' into add_CORS 2021-10-06 16:09:34 +03:00
Alexey Milovidov
b5230a5fe2 Web UI: render bars in table cells 2021-10-06 03:59:16 +03:00
Filatenkov Artur
8df228e6b8
Merge branch 'master' into add_CORS 2021-10-04 10:55:09 +03:00
Alexey Milovidov
135eac5aea Add logo 2021-10-03 09:08:01 +03:00
Maksim Kita
87f99c5104
Merge pull request #29643 from azat/listen_backlog 
Increase listen_backlog by default (to match default in newer linux kernel)
 2021-10-02 22:06:30 +03:00
Maksim Kita
395119339f Merge branch 'master' into rename-common 2021-10-02 17:15:10 +03:00
Maksim Kita
c966806205
Merge pull request #29529 from kitaisreal/external-xml-loaders-reload-with-server-configuration-support 
ExternalXMLLoaders reload with server configuration support
 2021-10-02 16:26:28 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
Azat Khuzhin
9d69786a6b Increase listen_backlog by default (to match default in newer linux kernel) 
It was 64 for a long time, and even linux kernel never has such a small
limit, it had 128 (net.core.somaxconn sysctl).

But recently, in 5.4, the default value had been increased even in
linux kernel, to 4096 [1].

  [1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=19f92a030ca6d772ab44b22ee6a01378a8cb32d4

Let's increase it in ClickHouse too.

Also note, that I've looked through some instances with
clickhouse-server only, and TcpExtListenOverflows was non zero (`nstat
-za`), in other words backlog of the listen socket indeed overflowed
there.

So it is better to increase the default to move the problem to
clickhouse-server itself (yes you will unlikely have 4K new incomming
connections at one time, with accept thread do not accept them all, but
still seems that it is possible, maybe due to some locks or something
else).
 2021-10-01 21:03:59 +03:00
Maksim Kita
4a480699bd Added tests 2021-10-01 16:44:09 +03:00