MovElb
142c63e487
done rebase
2020-05-30 23:02:11 +03:00
MovElb
b4b5c90343
squash
2020-05-30 20:05:05 +03:00
Alexey Milovidov
496fed5ed3
Remove "system_tables_lazy_load" option
2020-05-30 17:24:09 +03:00
Ivan Blinkov
d154415a5b
adjust comments
2020-05-30 11:13:04 +03:00
Alexey Milovidov
b79020d0a8
Emit a warning if server was build in debug or with sanitizers
2020-05-30 00:41:33 +03:00
Ivan Blinkov
d033941399
try to fix merge issues
2020-05-29 23:20:28 +03:00
Ivan Blinkov
0386e526b2
grammar
2020-05-29 23:03:59 +03:00
Ivan Blinkov
95ee742ee7
merge master
2020-05-29 22:56:10 +03:00
Ivan Blinkov
4ef322274d
Add integration test
2020-05-29 22:53:16 +03:00
Denis Glazachev
2863de750e
Merge branch 'master' into ldap-per-user-authentication
2020-05-28 01:30:52 +04:00
Denis Glazachev
d74f1357d4
Add LDAP authentication support
2020-05-28 01:06:33 +04:00
Ivan Blinkov
709b4f42c8
Prototype sending crash reports on segfaults
2020-05-27 22:11:04 +03:00
Ivan
5b858de374
Split programs/server into actual program and library ( #11186 )
...
* Split programs/server into actual program and library
2020-05-27 20:52:52 +03:00
Alexey Milovidov
7e1813825b
Return old names of macros
2020-05-24 01:24:01 +03:00
Alexey Milovidov
61cc605ee7
Remove old method
2020-05-24 01:21:29 +03:00
Alexey Milovidov
cd6d3c7509
Progress on task
2020-05-23 22:55:06 +03:00
Alexey Milovidov
f13ccfc8e8
Progress on task
2020-05-23 22:43:13 +03:00
Alexey Milovidov
7e2fb9ad65
Apply all transformations again
2020-05-23 22:38:30 +03:00
Alexey Milovidov
29762240de
Remove duplicate whitespaces (preparation)
2020-05-23 22:31:54 +03:00
Alexey Milovidov
86b4d5a86f
Remove duplicate whitespaces (preparation)
2020-05-23 22:09:06 +03:00
Alexey Milovidov
9d2a0d2dd7
Apply all transformations again
2020-05-23 21:59:49 +03:00
Alexey Milovidov
a2ad11897f
Remove duplicate whitespaces (preparation)
2020-05-23 21:53:58 +03:00
Alexey Milovidov
1f13515a65
Make all LOG in single line (preparation)
2020-05-23 21:31:37 +03:00
Alexey Milovidov
3a09f9ca1c
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_[^\W_]+\([^,]+, [^<+)"]+\);' | xargs sed -i -r -e 's/(LOG_[^\W_]+)\(([^,]+), ([^<+)"]+)\);/\1_FORMATTED(\2, \3);/'
2020-05-23 20:29:56 +03:00
Alexey Milovidov
f69cbdcbfc
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" \+ [^+]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" \+ ([^+]+)\);/\1_FORMATTED(\2, "\3{}", \4);/'
2020-05-23 20:09:37 +03:00
Alexey Milovidov
533f86278a
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5{}\7", \4, \6);/'
2020-05-23 20:00:41 +03:00
Alexey Milovidov
8042e5febe
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}\5{}", \4, \6);/'
2020-05-23 19:58:15 +03:00
Alexey Milovidov
e391b77d81
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5", \4);/'
2020-05-23 19:56:05 +03:00
Alexey Milovidov
ee4ffbc332
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}", \4);/'
2020-05-23 19:47:56 +03:00
Alexey Milovidov
8d2e80a5e2
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+"\)' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+, "[^"]+")\)/\1_FORMATTED(\2)/'
2020-05-23 19:42:39 +03:00
Alexey Milovidov
1a67c89c26
Enable mlock by default
2020-05-22 14:31:27 +03:00
Nikolai Kochetov
f7456f8320
Merge pull request #11004 from ClickHouse/remove-experimental-use-processors-flag-2
...
Remove allow_processors flag from executeQuery()
2020-05-22 10:47:15 +03:00
alexey-milovidov
2d9433242e
Merge pull request #11097 from ClickHouse/azat-memory-tracking-2
...
Merging #11022
2020-05-21 20:19:59 +03:00
alexey-milovidov
765510d5c3
Merge pull request #10991 from ClickHouse/enable-clang-tidy-for-programs-and-utils
...
Enable clang-tidy for programs and utils
2020-05-21 15:12:07 +03:00
Azat Khuzhin
09474ebea5
Fix memory tracking for two-level GROUP BY when not all rows read from Aggregator (TCP)
...
Example of such cases:
- SELECT GROUP BY LIMIT
- SELECT GROUP BY with subsequent MEMORY_LIMIT_EXCEEDED error
And it should be two-level aggregation, since otherwise there will be
only one hashtable which will be cleared correctly, only if you have
two-level GROUP BY some of hashtables will not be cleared since nobody
consume rows.
Before this patch:
09:39.015292 [ 3070801 ] {609a0610-e377-4132-9cf3-f49454cf3c96} <Information> executeQuery: Read 1000000 rows, 7.63 MiB in 0.707 sec., 1413826 rows/sec., 10.79 MiB/sec.
09:39.015348 [ 3070801 ] {609a0610-e377-4132-9cf3-f49454cf3c96} <Debug> MemoryTracker: Peak memory usage (for query): 51.93 MiB.
09:39.015942 [ 3070801 ] {} <Trace> Aggregator: Destroying aggregate states <-- **problem**
09:39.017057 [ 3070801 ] {} <Trace> Aggregator: Destroying aggregate states <--
09:39.017961 [ 3070801 ] {} <Debug> MemoryTracker: Peak memory usage (for query): 51.93 MiB.
09:39.018029 [ 3070801 ] {} <Information> TCPHandler: Processed in 0.711 sec.
After this patch:
16:24.544030 [ 3087333 ] {79da208a-b3c0-48d4-9943-c974a3cbb6ea} <Information> executeQuery: Read 1000000 rows, 7.63 MiB in 0.599 sec., 1670199 rows/sec., 12.74 MiB/sec.
16:24.544084 [ 3087333 ] {79da208a-b3c0-48d4-9943-c974a3cbb6ea} <Debug> MemoryTracker: Peak memory usage (for query): 72.11 MiB.
16:24.544398 [ 3087333 ] {79da208a-b3c0-48d4-9943-c974a3cbb6ea} <Trace> Aggregator: Destroying aggregate states
16:24.545485 [ 3087333 ] {79da208a-b3c0-48d4-9943-c974a3cbb6ea} <Trace> Aggregator: Destroying aggregate states
16:24.547053 [ 3087333 ] {} <Debug> MemoryTracker: Peak memory usage (for query): 72.11 MiB.
16:24.547093 [ 3087333 ] {} <Information> TCPHandler: Processed in 0.603 sec.
2020-05-21 02:06:55 +03:00
Nikolai Kochetov
1183bad4d0
Fix build.
2020-05-20 22:01:36 +03:00
alexey-milovidov
07924f0ae1
Merge pull request #11029 from azat/system_tables_lazy_load
...
Disable system_tables_lazy_load by default
2020-05-20 21:15:09 +03:00
alesapin
a416813597
Merge pull request #10777 from NanoBjorn/refactor-reservations
...
VolumePtr instead of DiskPtr in MergeTreeData*
2020-05-19 13:27:11 +03:00
Azat Khuzhin
ef01eb954f
Add note about system_tables_lazy_load into config.xml
2020-05-19 02:41:18 +03:00
Artem Zuikov
97c7447130
Bitonic sort improvements ( #10934 )
2020-05-19 00:41:23 +03:00
alexey-milovidov
551fbf8519
Update HTTPHandlerFactory.cpp
2020-05-18 22:46:32 +03:00
Artem Zuikov
bfcbc08bba
arcadia sync fixes
2020-05-18 15:50:23 +03:00
Artem Zuikov
0fe723acc9
Try to fix arcadia sync ( #10959 )
2020-05-18 13:26:23 +03:00
alesapin
07cb21ccb7
Merge branch 'master' into refactor-reservations
2020-05-18 11:43:48 +03:00
Alexey Milovidov
f54435e7fd
Fix clang-tidy
2020-05-18 11:08:55 +03:00
Alexey Milovidov
f4ac1eec04
Merge branch 'master' into quota-key-in-client
2020-05-17 08:15:13 +03:00
Gleb Novikov
1a25ac6e1f
Merge branch 'master' into refactor-reservations
2020-05-16 23:34:45 +03:00
Nikolai Kochetov
caab379602
Merge pull request #10932 from ClickHouse/pulling-executor
...
Add PullingPipelineExecutor.
2020-05-16 23:09:18 +03:00
Nikolai Kochetov
6ac93e9a61
Try simplify code.
2020-05-15 22:20:02 +03:00
Nikolai Kochetov
83a35a7390
try fix tests.
2020-05-15 20:41:12 +03:00
alesapin
9ae37a054c
Fix build without openCL
2020-05-15 11:31:32 +03:00
Artem Zuikov
8e8a2a17d6
build fixes
2020-05-15 03:01:31 +03:00
Ri
fc7afaa639
Bitonic sort on GPU (OpenCL) ( #10232 )
2020-05-15 03:01:14 +03:00
Nikolai Kochetov
681955661f
Add PullingPipelineExecutor.
2020-05-15 00:03:38 +03:00
alexey-milovidov
ddc84163a7
Merge pull request #10486 from azat/dist-send-on-INSERT
...
Fix distributed send that are scheduled by INSERT query
2020-05-11 06:28:35 +03:00
Gleb Novikov
57b6fb6d80
Merge branch 'master' into refactor-reservations
2020-05-10 13:01:45 +03:00
Alexey Milovidov
894a2ccb4b
Merge branch 'master' into even-more-warnings
2020-05-10 06:07:50 +03:00
Alexey Milovidov
9872964290
Enable "allow_introspection_functions" for development
2020-05-10 04:18:16 +03:00
Alexey Milovidov
4a13aba8de
Fix build
2020-05-10 02:34:24 +03:00
Alexey Milovidov
a576a4fbcd
Enable extra warnings for base, utils, programs
2020-05-10 01:59:34 +03:00
Gleb Novikov
390b39b272
VolumePtr instead of DiskPtr in MergeTreeData*
2020-05-10 00:24:15 +03:00
Ivan
85d783c247
Poco contrib refactoring ( #10396 )
...
* Remove config_common.h
* Refactor libcpuid contrib
* Remove support for libcpuinfo
* Define USE_CPUID in Arcadia
* Refactor Poco libraries
2020-05-08 17:11:19 +03:00
Alexander Tokmakov
d35150ecf3
fix http readonly request stucks on client close
2020-05-05 23:32:34 +03:00
alexey-milovidov
114f09cfca
Merge pull request #10666 from NanoBjorn/refactor-volume
...
Volumes and storages refactoring
2020-05-05 16:25:26 +03:00
Gleb Novikov
c637d99e07
Volumes and storages refactoring:
...
1. Moved Volume to separate file
2. Created IVolume interface and implemented current behaviour in implementation of new interface — VolumeJBOD
3. Replaced all old volume usages with new VolumeJBOD. Where it is unnecessary to have JBOD — left just IVolume.
4. Removed old Volume completely
5. Moved StoragePolicy to separated files
6. Moved DiskSelector to separated files
7. Removed DiskSpaceMonitor file
2020-05-04 23:15:38 +03:00
Alexey Milovidov
99c18c5a09
Check for suspicious codecs #4966
2020-05-04 03:11:49 +03:00
Alexey Milovidov
bb6eabd82d
Fix HTTP response code for some parse errors #10636
2020-05-03 22:47:07 +03:00
Azat Khuzhin
8ccaf1bf32
Add internal_replication example config.xml
2020-05-03 14:46:45 +03:00
Alexey Milovidov
c40ffb88c6
Comments in config.xml; reset values for production use case
2020-05-01 22:18:35 +03:00
Alexey Milovidov
600b396f1b
Merge remote-tracking branch 'origin/master' into sampling-memory-profiler
2020-05-01 21:48:20 +03:00
Alexander Kazakov
e9baaa439b
Implementation of new system metrics provider (Procfs) ( #10544 )
...
* New metrics provider (Procfs) + Refactored TasksStatsCounters
* Trivial statless test that ProcFS is provided
* Trivial perf test for ProcfsMetricsProvider
Co-authored-by: alexey-milovidov <milovidov@yandex-team.ru>
2020-05-01 21:47:41 +03:00
Alexey Milovidov
293ae88e7f
Add sampling memory profiler
2020-04-30 16:25:17 +03:00
alexey-milovidov
cef5f12186
Merge pull request #10542 from azat/logger.levels.logger
...
Ability to set logger level for loggers that have non allowed in xml tags symbols
2020-04-29 00:13:52 +03:00
tavplubix
11c3493676
Merge pull request #10547 from ClickHouse/zhang2014/feature/ISSUES-5436
...
Merging #7572
2020-04-28 12:40:48 +03:00
Alexander Tokmakov
aeac8cb621
fix
2020-04-28 10:19:37 +03:00
Alexander Tokmakov
9d43c2f6c2
save compiled regex
2020-04-28 02:05:15 +03:00
Alexander Tokmakov
e72a484beb
minor fixes
2020-04-28 02:05:15 +03:00
Azat Khuzhin
f767309574
Ability to set logger level for loggers that have non allowed in xml tags symbols
2020-04-27 21:30:29 +03:00
Azat Khuzhin
9ff0aaac9c
Add example of logger.levels usage into config.xml
2020-04-27 21:21:01 +03:00
zhang2014
d56002b81f
ISSUES-5436 fix ya make
2020-04-27 20:17:16 +08:00
zhang2014
81230944a4
ISSUES-5436 add integration test
2020-04-27 18:49:32 +08:00
Alexander Tokmakov
04d6b59ac0
Merge branch 'master' into database_atomic
2020-04-23 17:31:37 +03:00
zhang2014
0070f75218
ISSUES-5436 fix integration test failure & add test
2020-04-23 17:11:20 +08:00
Alexey Milovidov
07dcf40a6a
Simple server wide memory profiler
2020-04-22 20:52:21 +03:00
Alexander Tokmakov
fefbbd37df
Merge branch 'master' into database_atomic
2020-04-22 17:02:30 +03:00
Alexey Milovidov
0482e2e3ea
Implement suggestion from Sergey Veletsky
2020-04-22 01:40:18 +03:00
alexey-milovidov
8c2839d3c9
Merge pull request #10308 from ClickHouse/shared-context-lifetime
...
Shared context lifetime
2020-04-21 00:12:45 +03:00
Alexander Tokmakov
b29bddac12
Merge branch 'master' into database_atomic
2020-04-20 14:09:09 +03:00
zhang2014
318ab3b51e
ISSUES-5436 try fix build failure & pvs & style
2020-04-20 12:49:17 +08:00
Alexey Milovidov
1d885b3ff9
Merge branch 'master' into quota-key-in-client
2020-04-20 02:15:37 +03:00
Alexey Milovidov
2987e70aab
Introduce "max_server_memory_usage" setting
2020-04-20 01:59:21 +03:00
Alexey Milovidov
ec6b176f3b
Rework total memory tracker #10293
2020-04-20 01:59:21 +03:00
zhang2014
57cbecf935
ISSUES-5436 reworking predefine http
2020-04-20 03:18:36 +08:00
zhang2014
835dc4c445
After merge upsream master fix
2020-04-20 02:26:42 +08:00
zhang2014
b82eee5255
ISSUES-5436 fix review suggestions & add some '?' re2 test
2020-04-20 02:26:26 +08:00
zhang2014
8105a9bbe0
ISSUES-5436 fix review suggestions
2020-04-20 02:26:06 +08:00
tai
dafba9d66a
ISSUES-5436 add integration test for custom http
2020-04-20 02:25:43 +08:00
zhang2014
847f7ab439
ISSUES-5436 fix build failure & fix test failure
2020-04-20 02:25:43 +08:00
zhang2014
6b716e511b
ISSUES-5436 support custom http [part 6]
2020-04-20 02:25:12 +08:00
zhang2014
7aef95b082
ISSUES-5436 support custom http [part 5]
2020-04-20 02:23:18 +08:00
zhang2014
fd00757178
ISSUES-5436 support custom http [part 4]
2020-04-20 02:23:18 +08:00
zhang2014
60abff3341
ISSUES-5436 support custom http [part 3]
2020-04-20 02:23:18 +08:00
zhang2014
183eb82b7c
ISSUES-5436 support custom http [part 2]
2020-04-20 02:23:18 +08:00
zhang2014
159ba24fca
ISSUES-5436 support custom http
2020-04-20 02:23:18 +08:00
Nikolai Kochetov
84faa9af26
Merge branch 'master' into shared-context-lifetime
2020-04-17 19:13:13 +03:00
Nikolai Kochetov
ce11662ab1
Fix build.
2020-04-17 15:58:52 +03:00
Nikolai Kochetov
d51a9c551b
Remove shared_ptr to SharedContext from Context.
2020-04-17 12:47:40 +03:00
alexey-milovidov
17e7d4d88a
Merge pull request #10307 from abyss7/arcadia-4
...
Changes for auto-sync with Arcadia
2020-04-17 05:08:34 +03:00
Nikolai Kochetov
025093d354
Fix build.
2020-04-16 17:51:33 +03:00
Nikolai Kochetov
d6db27e372
Modify context.
2020-04-16 17:37:38 +03:00
Ivan Lezhankin
e230632645
Changes required for auto-sync with Arcadia
2020-04-16 15:31:57 +03:00
alexey-milovidov
a92a6d8e63
Merge pull request #10274 from ClickHouse/avoid-error-message-on-rst
...
Avoid error message on RST
2020-04-15 17:33:35 +03:00
alexey-milovidov
2a48b0ecd7
Merge pull request #10184 from xzkostyan/feature-zookeeper-ssl
...
Optional secured communication between ClickHouse and Zookeeper
2020-04-15 07:09:14 +03:00
Alexey Milovidov
ddd840a81b
Avoid error message on RST
2020-04-15 06:56:42 +03:00
Alexey Milovidov
fe59524481
Implemented in a different way
2020-04-15 04:59:14 +03:00
Alexey Milovidov
c22f91db01
Allow to pass quota_key in clickhouse-client
2020-04-15 04:15:05 +03:00
Alexander Tokmakov
6c0d9723e0
Merge branch 'master' into database_atomic
2020-04-13 18:47:47 +03:00
Alexander Tokmakov
5e6d4b9449
Merge branch 'master' into database_atomic
2020-04-12 16:35:44 +03:00
BohuTANG
e7f399f652
MySQLHandler: max_allowed_packet returned by default when server setup with select variables
2020-04-12 13:37:39 +08:00
Vitaly Baranov
ca634f603a
Revert resetting the settings to their defaults after each query in TCPHandler.
2020-04-10 21:09:59 +03:00
Alexander Tokmakov
033d7f9b32
Merge branch 'master' into database_atomic
2020-04-10 17:12:09 +03:00
Vitaly Baranov
8a19f6d322
Merge pull request #10115 from azat/path.xml-access_control_path
...
Rewrite access_control_path in path.xml
2020-04-09 11:38:08 +03:00
Konstantin Lebedev
b666f60af8
Optional secured communication between ClickHouse and Zookeeper
2020-04-09 01:40:54 +03:00
BohuTANG
f48fdda678
Enhanced compatibility with native mysql-connector-java(JDBC) ( #10021 )
...
* Skip the `/* comments */ SELECT @@variables ...` from mysql-connector-java setup for MySQL Handler #9336
mysql-connector setup query:
/* mysql-connector-java-5.1.38 ( Revision: ${revinfo.commit} ) */SELECT @@session.auto_increment_increment AS auto_increment_increment, @@character_set_client AS character_set_client, @@character_set_connection AS character_set_connection, @@character_set_results AS character_set_results, @@character_set_server AS character_set_server, @@init_connect AS init_connect, @@interactive_timeout AS interactive_timeout...
ClickHouse side Error:
{} <Error> executeQuery: Code: 62, e.displayText() = DB::Exception: Syntax error: failed at position 74: @@session.auto_increment_increment AS auto_increment_increment, @@character_set_client AS character_set_client, @@character_set_connection AS character_set_conn. Expected one of: CAST, NULL...
Client side Exception:
java.sql.SQLException: Syntax error: failed at position 74: @@session.auto_increment_increment AS auto_increment_increment, @@character_set_client AS character_set_client, @@character_set_connection AS character_set_conn. Expected one of: CAST...
* add repalce 'SHOW VARIABLES' for mysql-connector-java-5.1.34 #9336
* Add java client(JDBC) integration test to test_mysql_protocol
* shift out java tests from dbms
* Update MySQLHandler.cpp
* Update MySQLHandler.cpp
* test_mysql_protocol: add Test.java exit code 1 when expection
Co-authored-by: alexey-milovidov <milovidov@yandex-team.ru>
2020-04-09 00:52:19 +03:00
Alexander Tokmakov
dd1590830b
Merge branch 'master' into database_atomic
2020-04-08 22:00:46 +03:00
Azat Khuzhin
ed25ac8b12
Rewrite access_control_path in path.xml
...
Follow-up for: #9811
2020-04-08 10:45:14 +03:00
Vitaly Baranov
bc3e2cb48c
Add the option "access_control_path" to server config.
2020-04-07 23:31:59 +03:00
Vitaly Baranov
b71ff6160a
Add the "access_management" option to users.xml which enables a user to manage access rights.
...
The option is turned off by default.
2020-04-07 23:31:54 +03:00
Alexander Tokmakov
9c67d2716e
Merge branch 'master' into database_atomic
2020-04-03 03:19:11 +03:00
Ivan
97f2a2213e
Move all folders inside /dbms one level up ( #9974 )
...
* Move some code outside dbms/src folder
* Fix paths
2020-04-02 02:51:21 +03:00