thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-23 08:02:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
67,091 Commits 9,166 Branches 1,781 Tags 2.1 GiB
80335713cc
Commit Graph

211 Commits

Author SHA1 Message Date
alesapin
173b6fefdf Merge branch 'master' into in_memory_raft 2021-01-25 12:03:28 +03:00
Alexey Milovidov
8d335b1490 Allow to enable or disable watchdog via env var 2021-01-23 23:58:36 +03:00
alesapin
c1e36cfe70 Something working 2021-01-22 19:04:57 +03:00
Alexey Milovidov
950bfb3ec6 Allow change max_server_memory_usage without restart 2021-01-16 18:33:46 +03:00
Alexey Milovidov
cea1b7dba5 Fix build 2021-01-07 16:37:06 +03:00
Alexey Milovidov
6f481d7512 Add integrity checks for ClickHouse binary 2021-01-07 05:56:57 +03:00
John Skopis
9bcc5a6175 Support interserver credential rotation 
Restarting a server instance to change the interserver password results
in many replicas being out of sync until all clusters are using the new
credential.

This commit adds dynamic credential loading for both the client
(Replicated* tables) and server (InterserverIOHTTPHandler).

This commit also adds the ability to rotate credentials, i.e. accept more
than one credential during a credential change.

state0 (no auth):

    <interserver_http_credentials />

state1 (auth+allow_empty migration):

    <interserver_http_credentials>
        <user>admin</user>
        <password>222</password>
        <allow_empty>true</allow_empty>
    </interserver_http_credentials>

state2 (auth+new admin password migration):

    <interserver_http_credentials>
        <user>admin</user>
        <password>333</password>
        <users>
            <admin>222</admin>
        </users>
    </interserver_http_credentials>
 2021-01-04 09:59:09 +00:00
Alexey Milovidov
fdc8950fbe Do not use watchdog when server is run from tty 2020-12-23 17:31:10 +03:00
alexey-milovidov
29ad7b6d94
Merge pull request #13516 from ClickHouse/watchdog 
If server was terminated by OOM killer, print message in log.
 2020-12-23 05:35:44 +03:00
Alexey Milovidov
f9012b12fb Proper implementation 2020-12-22 06:01:51 +03:00
Alexey Milovidov
932ade2cb8 Merge branch 'master' into watchdog 2020-12-22 03:10:15 +03:00
nikitamikhaylov
8501c7a831 better 2020-12-22 02:03:08 +03:00
nikitamikhaylov
cd3a73f9d3 done 2020-12-22 00:47:10 +03:00
Amos Bird
c447628d42
Add connection thread metrics 2020-12-18 15:22:55 +08:00
Alexey Milovidov
f7f8ff8758 Merge branch 'master' into sigaltstack 2020-12-17 22:16:30 +03:00
Alexey Milovidov
7f280bef37 Merge branch 'master' into sigaltstack 2020-12-17 05:09:10 +03:00
alesapin
ea4d11cb73 Wait for connections to special servers 2020-12-16 13:04:46 +03:00
Alexander Kuzmenkov
fb1221148d
Merge pull request #17748 from azat/custom-TLD 
Add ability to use custom TLD list
 2020-12-11 12:42:19 +03:00
Azat Khuzhin
840a21d073 Add top_level_domains_path for easier overriding 2020-12-09 21:08:31 +03:00
Azat Khuzhin
916cbd6610 Add ability to use custom TLD list 
v2: Add a note that top_level_domains_lists aren not applied w/o restart
v3: Remove ExtractFirstSignificantSubdomain{Default,Custom}Lookup.h headers
v4: TLDListsHolder: remove FIXME for dense_hash_map (this is not significant)
 2020-12-09 21:08:22 +03:00
alesapin
0dd9c720f4 Better server 2020-12-09 11:58:41 +03:00
alesapin
758dcd1972 Separate factory for test keeper handlers 2020-12-09 11:45:36 +03:00
alesapin
3d0dbbe411 Fix build 2020-12-08 17:49:18 +03:00
Alexey Milovidov
3e2447391b Support for PROXY protocol 2020-12-03 00:05:51 +03:00
alesapin
fb86eaf6fc Better errors and better config 2020-11-25 16:19:09 +03:00
alesapin
310307fa72 Replace eventfd with pipes 2020-11-25 11:18:15 +03:00
alesapin
4d40d9be3f Merge branch 'master' into complete_zk_api 2020-11-25 10:31:16 +03:00
Vitaly Baranov
c2edd9f8ce Rework async server: switch to coroutine-like approach, no using statuses anymore for control flow. 2020-11-24 17:53:34 +03:00
Vitaly Baranov
ff62fd4967 Add an adapter for protocol servers. 2020-11-24 17:53:34 +03:00
Vitaly Baranov
44717797c9 Fix code style and compilation. 2020-11-24 17:53:34 +03:00
mnkonkova
6cd1557d67 Implement GRPC protocol. 2020-11-24 17:53:34 +03:00
alesapin
1de175257b More const 2020-11-23 23:22:04 +03:00
alesapin
5a48076802 Trying to get rid of leak 2020-11-23 13:55:00 +03:00
alesapin
0208751a7d Add dirty hack for connections close 2020-11-20 15:46:05 +03:00
alesapin
991386e37b Fix stupid error 2020-11-19 13:56:36 +03:00
alesapin
4aad6a597c Merge branch 'correctly_send_close_request' into complete_zk_api 2020-11-12 17:46:02 +03:00
tavplubix
058aa8f85e
Merge pull request #16824 from ClickHouse/replace_stringstreams_with_buffers 
Replace std::*stringstreams with DB::*Buffers
 2020-11-12 01:11:44 +03:00
alesapin
6398271432 Merge branch 'master' into complete_zk_api 2020-11-11 16:57:24 +03:00
alesapin
d83c68fca8 Fix timeouts 2020-11-11 16:55:28 +03:00
alesapin
66236d6ebb Refactored a little 2020-11-11 16:07:06 +03:00
Alexander Tokmakov
b94cc5c4e5 remove more stringstreams 2020-11-10 21:22:26 +03:00
alesapin
932ea0ef8b Merge branch 'master' into complete_zk_api 2020-11-10 13:56:11 +03:00
Amos Bird
aa8e6db786
reload auxiliary zookeepers configuration 2020-11-10 15:56:37 +08:00
alesapin
c2525ef211 Server and client pinging each other 2020-10-30 17:16:47 +03:00
Alexey Milovidov
2e0a979e3a Fix inconsistency in FormatFactory 2020-10-29 06:39:43 +03:00
Alexey Milovidov
068235c2d7 Disable alt stack for sanitizers 2020-10-26 05:18:36 +03:00
Azat Khuzhin
72d7b6117e Use total_memory_tracker when there is no other MemoryTracker object. 
This should significantly reduce the MemoryTracking drift, test shows
that there is 0 drift after query storm (100 queries, via http/tcp/tcp
in one session).

TL;DR;

To track memory, clickhouse creates memory tracker object for each
thread **explicitly**, but until it is not created the memory
allocations are not under account.
There should not be lot of allocations w/o memory tracker, since most of
the time it is created early enough, but even this maybe enough to
trigger some problems.

Plus sometimes it is not possible to create it, for example some 3d
party library does not allow to do this explicitly:
- for example before #15740 allocations from librdkafka threads,
- or even worse, poco threads, they don't have any routines to do this.
This won't be a problem for `MemoryTracking` metric if the deallocation
will be done from the same thread w/o memory tracker (or vise versa),
but this is not always true.

NOTE, that this will slow down per-thread allocations w/o memory
tracker, since before this patch there were no memory tracking for them
while now they will be accounted in total_memory_tracker, and for
total_memory_tracker max_untracked_memory is always reached.
But this should not be significant.
 2020-10-23 21:07:52 +03:00
Alexey Milovidov
7d97c8b053 Remove useless code 2020-10-07 11:21:32 +03:00
tavplubix
4578d89819
Merge pull request #15348 from ClickHouse/replicate_default_args 
Continuation of #14791
 2020-09-28 18:53:02 +03:00
Alexander Tokmakov
34addcf61f add review suggestions 2020-09-26 22:18:28 +03:00
alesapin
97671a1771 Ugly fix for default database race 2020-09-25 14:27:00 +03:00
Alexey Milovidov
1f6e55ff35 Allow to run without /proc/self/maps 2020-09-25 05:03:58 +03:00
Alexander Kuzmenkov
679c0988a5 Merge remote-tracking branch 'origin/master' into HEAD 2020-09-22 16:25:52 +03:00
Alexander Kuzmenkov
dde4cf70e1 Merge remote-tracking branch 'origin/master' into HEAD 2020-09-22 14:03:59 +03:00
Alexander Kuzmenkov
478c7309d4
Merge pull request #14843 from ClickHouse/aku/global-pool 
Exception on double init of global thread pool
 2020-09-22 13:44:39 +03:00
alexey-milovidov
3e0509af69
Merge pull request #13573 from amosbird/rms 
specific ReplicatedMergeTree settings
 2020-09-21 01:36:33 +03:00
alexey-milovidov
1474b80ecc
Merge pull request #15008 from ClickHouse/clang-tidy-11-without-enabling 
Fix terribly wrong code
 2020-09-20 02:00:54 +03:00
alexey-milovidov
c461d782e4
Merge pull request #14929 from ClickHouse/mlock-code-segment 
Only mlock code segment
 2020-09-19 13:46:49 +03:00
Alexey Milovidov
ef5363b87f Fix terribly wrong code 2020-09-19 02:05:13 +03:00
alexey-milovidov
988b20a32c
Merge pull request #14684 from azat/parallel-distributed_ddl 
Allow parallel execution of distributed DDL
 2020-09-18 22:18:17 +03:00
Alexander Kuzmenkov
449189dcda Initialize global thread pool before we fetch configs from ZK 2020-09-18 13:47:09 +03:00
Alexey Milovidov
eb523f1a7c Merge branch 'master' into mlock-code-segment 2020-09-17 21:15:09 +03:00
Alexander Kuzmenkov
fb64cf210a straighten the protocol version 2020-09-17 17:37:29 +03:00
Alexey Milovidov
068e8576b7 Corrections 2020-09-17 15:53:52 +03:00
Alexey Milovidov
8bce20076c Only mlock code segment 2020-09-17 15:39:37 +03:00
sundy-li
544b2cb20d add configChanged method for zookeeper 
fix logic error && skip reload testkeeper
 2020-09-17 13:33:45 +08:00
Alexey Milovidov
2ac88ab47d Added config option 2020-09-14 21:08:09 +03:00
Azat Khuzhin
dd867b787f Allow parallel execution of distributed DDL 
Add distributed_ddl.pool_size to control maximum parallel to handle
distributed DDL.

Also:
- convert Exception constructors to fmt-like
- use sleepFor* over std::this_thread::sleep_for()
 2020-09-12 02:32:08 +03:00
Alexey Milovidov
a644733139 Attempt to make performance test more reliable 2020-09-10 12:05:57 +03:00
Yatsishin Ilya
45e54f81c7 better resolv.conf, add DNSCacheUpdater logs 2020-09-02 11:07:46 +03:00
Amos Bird
05a5a13e08
specific ReplicatedMergeTree settings 2020-08-27 22:37:41 +08:00
Vitaly Baranov
0759dff12b Support <user_directories> section in the main config. 2020-08-16 19:15:38 +03:00
Vitaly Baranov
2909ed1bc0 Better initialization of access storages. Make list of access storages dynamic. 2020-08-16 19:15:34 +03:00
Alexey Milovidov
e43746395e Merge branch 'master' into codespell-2 2020-08-16 14:57:21 +03:00
Alexey Milovidov
6bc8da633c Minor modification 2020-08-16 14:52:55 +03:00
Alexey Milovidov
8e36bfe54d Merge branch 'master' into watchdog 2020-08-16 14:37:55 +03:00
Alexander Tokmakov
a6ff049eec use Atomic for system database 2020-08-12 23:40:13 +03:00
Alexey Milovidov
6c4df0f27a Better tool 2020-08-08 17:10:49 +03:00
Alexey Milovidov
ae716e13e0 Watchdog (experimental) 2020-08-08 07:52:09 +03:00
Alexey Milovidov
3b8020168f Add simple watchdog 2020-08-08 07:44:04 +03:00
Vitaly Baranov
1a4a8a219c
Merge pull request #13305 from vitlibar/correct-error-message-if-setting-not-found-in-users_xml 
Correct error message if setting not found in users.xml
 2020-08-07 23:31:23 +03:00
Vitaly Baranov
a804f9499d Use references while iterating through settings. 2020-08-04 04:00:38 +03:00
Vitaly Baranov
dadebadcac Print correct error message in log for unknown settings in users.xml 2020-08-04 00:20:33 +03:00
Alexey Milovidov
778abb346f Merge branch 'master' into merge-tree-settings-sanity-check 2020-08-02 17:13:17 +03:00
Vitaly Baranov
7c4ae5ee65 Add the parameter custom_settings_prefixes to the server config. 2020-07-31 20:57:49 +03:00
Vitaly Baranov
56665a15f7 Rework and rename the template class SettingsCollection => BaseSettings. 2020-07-31 20:54:18 +03:00
Alexey Milovidov
b9f49d31df Sanity checks for MergeTreeSettings 2020-07-30 22:08:13 +03:00
Denis Glazachev
8688a1f5d0 Recreate ExternalAuthenticators (LDAP) on config update 2020-07-10 15:59:48 +04:00
alesapin
1aa45f203b
bump CI 2020-07-08 13:40:02 +03:00
Denis Glazachev
9effacfbc1 Merge branch 'master' into ldap-per-user-authentication 
* master: (1102 commits)
  Update README.md
  Update README.md
  Update README.md
  Update index.md
  [docs] add intrdocution for statements page (#12189)
  Revert "Run perf tests with memory sampling (for allocations >1M)"
  Sanitize LINK_LIBRARIES property for the directories (#12160)
  [docs] refactor Domains overview (#12186)
  DOCS-647: toStartOfSecond (#12190)
  [docs] add intrdocution for commercial page (#12187)
  DOCSUP-1348 Russian translation for new functions (#133) (#12194)
  changelog fixes
  Update index.md (#12191)
  Update zh kafka.md title (#12192)
  Added test for #3767
  style fix for #12152
  Tests for fixed issues #10846 and #7347
  changelog fixes
  [docs] introduction for special table engines (#12170)
  [docs] introduction for third-party interfaces (#12175)
  ...

# Conflicts:
#	src/Access/ya.make
#	src/Common/ErrorCodes.cpp
 2020-07-08 00:42:09 +04:00
Alexey Milovidov
176a7f2f72 Normalize "pid" file handling #3501 2020-07-04 16:54:24 +03:00
Alexey Milovidov
44f2742a51 Fix bad log message at server startup 2020-06-27 15:56:06 +03:00
Alexander Kuzmenkov
d77f397b38 review fixes 2020-06-26 03:18:33 +03:00
Alexander Kuzmenkov
593a0181bd
Merge pull request #11616 from ClickHouse/aku/perf-benchmark 
Add concurrent benchmark to performance test
 2020-06-25 09:36:52 +03:00
Alexander Kuzmenkov
e0bdbe73d2 Merge remote-tracking branch 'origin/master' into HEAD 2020-06-23 15:31:09 +03:00
Alexander Kuzmenkov
ab809f59b9 memory usage settings 2020-06-23 15:30:45 +03:00
Alexander Kuzmenkov
96d2e9c997 Initialize GlobalThreadPool explicitly 2020-06-22 22:04:12 +03:00
alexey-milovidov
4ee623ccac
Merge pull request #10242 from MovElb/movelb-postgresql-wire-protocol-impl 
PostgreSQL wire protocol implementation
 2020-06-21 14:39:22 +03:00
Denis Glazachev
5db60202b6 Merge branch 'master' into ldap-per-user-authentication 2020-06-19 00:11:08 +04:00
Alexey Milovidov
bb6c0743fc Change the level of log message about failure to listen, to warning #4406 2020-06-15 23:30:36 +03:00