thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 19:02:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
152,840 Commits 9,326 Branches 1,783 Tags 2.1 GiB
6bf5c092bb
Commit Graph

2026 Commits

Author SHA1 Message Date
alexey-milovidov
0f706c01ca
Merge pull request #13888 from vladimir-golovchenko/add-date-trunc-function 
Added date_trunc function
 2020-08-27 02:12:27 +03:00
vladimir golovchenko
f3cfd39003 Fixed .gitignore to exclude tests-generated data. 2020-08-25 16:41:23 -07:00
Alexander Kuzmenkov
05ad9b9fff opentelemetry wip 2020-08-20 23:59:58 +03:00
Denis Glazachev
ec52a165af Style fixes 2020-08-20 12:46:42 +04:00
Denis Glazachev
bdfea652c1 Change user_template to roles 
Change top_enclosing_storage to access_control_manager
Simplify the lookup in peer storages
 2020-08-20 11:39:27 +04:00
Denis Glazachev
58f73ff041 Merge branch 'master' into ldap-any-user-authentication 
* master: (30 commits)
  Documentation improve:  Translate [select] section into Chinese to improve readability (#13814)
  Update adopters.md
  partially disable test with MaterializeMySQL
  one more unroll for arerage.cpp
  Update adopters.md
  Update adopters.md
  Update adopters.md
  Update adopters.md
  fix #13819
  DOCSUP-1888: Documented the input_format_avro_allow_missing_fields setting (#13671)
  Fix "Arcadia" and "Unbundled" builds
  DOCSUP-928: Documented the groupArraySample function (#13791)
  Documentation about ReplacingMergeTree extended with type DateTime64 for column (#13498)
  doc: update quotas.md (#13400)
  Fix 01356_initialize_aggregation in unbundled build (change topKWeighted order)
  Fix style
  Add normalizedQueryHash function with tests
  Fix topK/topKWeighted merge (wtih non-default parameters)
  Add test
  Add function "normalizeQuery"
  ...

# Conflicts:
#	programs/server/Server.cpp
#	src/Access/AccessControlManager.cpp
#	src/Access/AccessControlManager.h
 2020-08-18 14:54:02 +04:00
Denis Glazachev
fbbaf645b9 Merge branch 'master' into ldap-any-user-authentication 
* master: (70 commits)
  Fix terrible markup due to wrong @blinkov's script.
  Update extended-roadmap.md
  Update extended-roadmap.md
  Fix bug on the website
  Suppress Shellcheck
  Fix typos
  Fix typos
  Install codespell
  Fix typo
  Update gtest_thread_pool_schedule_exception.cpp
  Fix flaky unit test
  Fix set index with const column pred
  Updating healthcheck parameters.
  Fix step overflow in range()
  * Moving to standard healthcheck parameters. * Adding docker-compose pull and docker-compose down before executing docker-compose up. * Adding docker-compose ps if any of the containers are unhealthy.
  Fix topK test
  Adding missing certificates and dhparam.pem for openldap4.
  Keep "metadata_loading" test only for release build
  Fix error in test
  Fix typo
  ...
 2020-08-17 18:08:24 +04:00
Vitaly Baranov
0759dff12b Support <user_directories> section in the main config. 2020-08-16 19:15:38 +03:00
Vitaly Baranov
2909ed1bc0 Better initialization of access storages. Make list of access storages dynamic. 2020-08-16 19:15:34 +03:00
Alexey Milovidov
e43746395e Merge branch 'master' into codespell-2 2020-08-16 14:57:21 +03:00
Alexey Milovidov
6bc8da633c Minor modification 2020-08-16 14:52:55 +03:00
Alexey Milovidov
8e36bfe54d Merge branch 'master' into watchdog 2020-08-16 14:37:55 +03:00
Denis Glazachev
7375dc5d66 Merge branch 'master' into ldap-any-user-authentication 
* master: (956 commits)
  Remove ZooKeeper from unit tests
  Revert check location of Docker compose files @qoega.
  Update developer-instruction.md
  trigger the CI
  Update settings.md
  Update settings.md
  Fix
  fix sync 2
  Fix tests.
  Add docker for style check
  fix sync 1
  passwd and group location error
  Hotfix for pushdown with StorageMerge (#13679)
  Fix error with batch aggregation and -Array combinator
  Fix cassandra build on macos
  Update adopters.md
  Update adopters.md
  Fix build after merge
  Fix shared build
  log error message
  ...

# Conflicts:
#	tests/testflows/ldap/docker-compose/clickhouse-service.yml
 2020-08-15 14:18:40 +04:00
Alexander Tokmakov
a6ff049eec use Atomic for system database 2020-08-12 23:40:13 +03:00
Alexey Milovidov
c7a6a18a75 Preparation to enable clickhouse install on Mac and FreeBSD 2020-08-10 02:58:09 +03:00
Alexey Milovidov
4129ad5bbf Merge branch 'embed-configs' into clickhouse-install 2020-08-09 05:17:12 +03:00
Alexey Milovidov
aa3b4bbfe0 Fix build 2020-08-09 05:16:55 +03:00
Alexey Milovidov
8d616444e4 Support for AArch64 2020-08-09 02:54:38 +03:00
Alexey Milovidov
182e2929f3 Merge branch 'master' into embed-configs 2020-08-09 02:41:44 +03:00
Alexey Milovidov
f459640269 Fix build 2020-08-09 02:41:34 +03:00
Alexey Milovidov
6c4df0f27a Better tool 2020-08-08 17:10:49 +03:00
Alexey Milovidov
ae716e13e0 Watchdog (experimental) 2020-08-08 07:52:09 +03:00
Alexey Milovidov
3b8020168f Add simple watchdog 2020-08-08 07:44:04 +03:00
Alexey Milovidov
0f79eb3cc5 Embed configs into binary 2020-08-08 06:42:42 +03:00
Alexey Milovidov
0cbbe153cd Fix typos, the last 1% 2020-08-08 04:21:04 +03:00
Vitaly Baranov
1a4a8a219c
Merge pull request #13305 from vitlibar/correct-error-message-if-setting-not-found-in-users_xml 
Correct error message if setting not found in users.xml
 2020-08-07 23:31:23 +03:00
alexey-milovidov
9cb0914bbd
Merge pull request #13447 from vladimir-golovchenko/fix-server-gitignore 
Updated gitignore-files.
 2020-08-07 05:01:33 +03:00
vladimir golovchenko
cb153d2605 Updated gitignore-files. 2020-08-06 18:05:32 -07:00
Vitaly Baranov
a804f9499d Use references while iterating through settings. 2020-08-04 04:00:38 +03:00
Vitaly Baranov
dadebadcac Print correct error message in log for unknown settings in users.xml 2020-08-04 00:20:33 +03:00
Alexey Milovidov
778abb346f Merge branch 'master' into merge-tree-settings-sanity-check 2020-08-02 17:13:17 +03:00
Vitaly Baranov
18b21511a9
Merge pull request #13013 from vitlibar/implement-custom-settings 
Implement custom settings
 2020-08-02 05:01:14 +03:00
Alexey Milovidov
53450b1b09 Merge branch 'master' into merge-tree-settings-sanity-check 2020-08-02 02:47:57 +03:00
Vitaly Baranov
7c4ae5ee65 Add the parameter custom_settings_prefixes to the server config. 2020-07-31 20:57:49 +03:00
Vitaly Baranov
56665a15f7 Rework and rename the template class SettingsCollection => BaseSettings. 2020-07-31 20:54:18 +03:00
Alexey Milovidov
c3ad710b84 Merge branch 'master' into crash-log 2020-07-31 16:12:53 +03:00
Alexey Milovidov
b9f49d31df Sanity checks for MergeTreeSettings 2020-07-30 22:08:13 +03:00
Denis Glazachev
479fa4c325 Improve LDAP-related comments 2020-07-27 14:24:56 +04:00
Denis Glazachev
90a064c7a6 Fix compilation 2020-07-24 19:39:18 +04:00
Denis Glazachev
3b3404c326 Style fix 
Remove unused declarations
 2020-07-23 22:10:57 +04:00
Denis Glazachev
5d6b5101fe Implement LDAPAccessStorage and integrate it into AccessControlManager 
Rename ExternalAuthenticators::setConfig to setConfiguration
Revisit LDAP servers config section comments
Add user_directories config section with comments (only for ldap)
Fix bug in MemoryAccessStorage::insertImpl
 2020-07-23 21:55:24 +04:00
Denis Glazachev
8688a1f5d0 Recreate ExternalAuthenticators (LDAP) on config update 2020-07-10 15:59:48 +04:00
Alexey Milovidov
31cbdd1a56 system.crash_log: development 2020-07-09 07:15:45 +03:00
alesapin
1aa45f203b
bump CI 2020-07-08 13:40:02 +03:00
Denis Glazachev
9effacfbc1 Merge branch 'master' into ldap-per-user-authentication 
* master: (1102 commits)
  Update README.md
  Update README.md
  Update README.md
  Update index.md
  [docs] add intrdocution for statements page (#12189)
  Revert "Run perf tests with memory sampling (for allocations >1M)"
  Sanitize LINK_LIBRARIES property for the directories (#12160)
  [docs] refactor Domains overview (#12186)
  DOCS-647: toStartOfSecond (#12190)
  [docs] add intrdocution for commercial page (#12187)
  DOCSUP-1348 Russian translation for new functions (#133) (#12194)
  changelog fixes
  Update index.md (#12191)
  Update zh kafka.md title (#12192)
  Added test for #3767
  style fix for #12152
  Tests for fixed issues #10846 and #7347
  changelog fixes
  [docs] introduction for special table engines (#12170)
  [docs] introduction for third-party interfaces (#12175)
  ...

# Conflicts:
#	src/Access/ya.make
#	src/Common/ErrorCodes.cpp
 2020-07-08 00:42:09 +04:00
Nikita Mikhaylov
5c6d6bdf54
Merge pull request #11995 from azat/load_balancing-priority 
Load balancing manual priority
 2020-07-06 13:32:20 +04:00
alexey-milovidov
f1151ed474
Merge pull request #12133 from ClickHouse/normalize-pid-file 
Normalize "pid" file handling #3501
 2020-07-04 21:36:27 +03:00
Alexey Milovidov
176a7f2f72 Normalize "pid" file handling #3501 2020-07-04 16:54:24 +03:00
alesapin
c57edd2018 Tiny fixes 2020-07-03 11:02:35 +03:00
alesapin
64583ceb22 Merge branch 'master' into add-storage-rabbitmq-read-only 2020-07-02 16:08:29 +03:00
Azat Khuzhin
ebff4eae7d Add replica priority for load_balancing 
Make default 1, to match with <weight>
 2020-06-29 23:03:28 +03:00
Azat Khuzhin
e547f6b6d7 Add <weight> example in <remote_servers> into dist config 2020-06-29 23:03:28 +03:00
Alexey Milovidov
44f2742a51 Fix bad log message at server startup 2020-06-27 15:56:06 +03:00
Alexander Kuzmenkov
d77f397b38 review fixes 2020-06-26 03:18:33 +03:00
alexey-milovidov
a34032cace
Merge pull request #11300 from blinkov/sentry 
Opt-in support for sending crash reports
 2020-06-25 17:20:50 +03:00
Alexander Kuzmenkov
593a0181bd
Merge pull request #11616 from ClickHouse/aku/perf-benchmark 
Add concurrent benchmark to performance test
 2020-06-25 09:36:52 +03:00
alesapin
3fc65b3269 Merge branch 'master' into kssenii-rabbit-mq 2020-06-24 20:14:28 +03:00
Alexander Kuzmenkov
f0fbba5522 allow implicit initialization 2020-06-23 20:31:46 +03:00
Denis Glazachev
d17d72b075 More generic DN in the example 2020-06-23 18:23:34 +04:00
Alexander Kuzmenkov
e0bdbe73d2 Merge remote-tracking branch 'origin/master' into HEAD 2020-06-23 15:31:09 +03:00
Alexander Kuzmenkov
ab809f59b9 memory usage settings 2020-06-23 15:30:45 +03:00
Alexander Kuzmenkov
96d2e9c997 Initialize GlobalThreadPool explicitly 2020-06-22 22:04:12 +03:00
Alexander Kuzmenkov
bd5ab9c686 Merge remote-tracking branch 'origin/master' into bnc/config-max-num-threads 2020-06-22 15:55:39 +03:00
Ivan Blinkov
2c0ff29c48 Merge branch 'master' of github.com:ClickHouse/ClickHouse into sentry 2020-06-22 10:06:36 +03:00
alexey-milovidov
4ee623ccac
Merge pull request #10242 from MovElb/movelb-postgresql-wire-protocol-impl 
PostgreSQL wire protocol implementation
 2020-06-21 14:39:22 +03:00
Denis Glazachev
5db60202b6 Merge branch 'master' into ldap-per-user-authentication 2020-06-19 00:11:08 +04:00
Ivan Blinkov
8b50e3450b move the default endpoint to config 2020-06-16 23:01:15 +03:00
Denis Glazachev
6338225c50 Fix tls_cipher_suite example 2020-06-16 17:49:18 +04:00
Ivan Blinkov
7e2bb1fd83 Merge branch 'master' of github.com:ClickHouse/ClickHouse into sentry 2020-06-16 16:02:55 +03:00
Ivan Blinkov
0e77692a27 improvements after review comments 2020-06-16 15:56:28 +03:00
Bharat Nallan
c43bd228ab make max global thread pool setting configurable 
This PR adds a server level config for overriding the default max number
of threads in global thread pool that is currently allowed (10,000).

This might be useful in scenarios where there are a large number of
distributed queries that are executing concurrently and where the
default number of max threads might not be necessarily be sufficient.
 2020-06-15 22:04:30 -07:00
Alexey Milovidov
bb6c0743fc Change the level of log message about failure to listen, to warning #4406 2020-06-15 23:30:36 +03:00
Alexey Milovidov
a421e7e4b4 Added a test 2020-06-14 00:13:52 +03:00
Denis Glazachev
276fcd8903 Add/rename parameters that control TLS 2020-06-12 21:59:47 +04:00
Ivan Blinkov
e98a91e9f7 Merge branch 'master' of github.com:ClickHouse/ClickHouse into sentry 2020-06-12 16:53:00 +03:00
Alexander Kuzmenkov
e354bbc78c
Merge pull request #11588 from ClickHouse/aku/async-log 
Add system.asynchronous_metric_log
 2020-06-11 16:10:35 +03:00
Ivan Blinkov
b70ed921ea Merge branch 'master' of github.com:ClickHouse/ClickHouse into sentry 2020-06-11 12:22:13 +03:00
kssenii
18820814f5 Merge 2020-06-10 23:25:33 +00:00
Denis Glazachev
9e3a28a6b8 Merge branch 'master' into ldap-per-user-authentication 
* master: (414 commits)
  Update file.md
  Update merge.md
  Update dictionary.md
  Update external-data.md
  Update distributed.md
  Update null.md
  Update set.md
  Update join.md
  Update url.md
  Update view.md
  Update materializedview.md
  Update memory.md
  Update buffer.md
  Update generate.md
  removed a sentence about global lock during rename (#11577)
  greatCircleAngle en translation (#11584)
  Update configuration-files.md
  try fix flacky test
  Update why.html
  Update rich.html
  ...

# Conflicts:
#	src/Common/ErrorCodes.cpp
#	utils/ci/jobs/quick-build/run.sh
 2020-06-11 03:06:17 +04:00
Denis Glazachev
848330b37a Expect <ldap_servers> in main config.xml 2020-06-11 02:48:15 +04:00
Alexander Kuzmenkov
a6648516f4 Add files; flush more often 2020-06-11 00:16:58 +03:00
Alexey Milovidov
fd3ff19868 Fix trivial error in log message #11399 2020-06-10 22:34:23 +03:00
Alexander Kuzmenkov
e4be52f35f Add system.asynchronous_metric_log 2020-06-10 22:17:54 +03:00
Ivan Blinkov
5b3fef526e Merge branch 'master' of github.com:ClickHouse/ClickHouse into sentry 2020-06-09 11:51:29 +03:00
Alexey Milovidov
df19db1509 Added a test for history in clickhouse-client 2020-06-07 20:29:34 +03:00
kssenii
68b94c5c20 Fixes 2020-06-05 13:46:20 +00:00
Alexey Milovidov
3d68cd4df6 Also check for users.xml elements in config.xml 2020-06-04 23:34:33 +03:00
Alexey Milovidov
02e14f9fe8 Also check for users.xml elements in config.xml 2020-06-04 23:33:19 +03:00
Alexey Milovidov
a89ce20d38 Added a check for incorrect settings 2020-06-04 22:30:30 +03:00
Denis Glazachev
c61cbe8e98 Merge branch 'master' into ldap-per-user-authentication 
* master: (114 commits)
  Update PushingToViewsBlockOutputStream.cpp
  Update PushingToViewsBlockOutputStream.cpp
  make clang-10 happy
  Fix sync_async test (remove timeout)
  CLICKHOUSEDOCS-631: temporary_files_codec, join_on_disk_max_files_to_merge settings. (#11242)
  Suppress output of cancelled queries in clickhouse-client #9473
  Better log messages in ConfigReloader
  fix select from StorageJoin
  Fix unit tests under MSan
  Added test.
  Fix build.
  Fix arguments for AggregateFunctionQuantile/
  Update style.md
  Add a guide on error messages.
  Report dictionary name on dictionary load errors.
  more types in ASOF JOIN (#11301)
  Fix part_log test
  Update test.
  Add perftest.
  Parallel processing for PushingToViewsBlockOutputStream::writeSuffix
  ...
 2020-06-02 12:42:29 +04:00
Ivan Blinkov
6bd4282727 Merge branch 'master' of github.com:ClickHouse/ClickHouse into sentry 2020-06-01 21:49:56 +03:00
alexey-milovidov
19c6e0e5ae
Merge pull request #11307 from ClickHouse/remove-system-tables-lazy-load 
Remove "system_tables_lazy_load" option
 2020-06-01 16:26:23 +03:00
Ivan Blinkov
fcc7ff0ef9 Merge branch 'master' of github.com:ClickHouse/ClickHouse into sentry 2020-06-01 08:42:03 +03:00
Alexey Milovidov
87cf123ef8 Merge branch 'master' into remove-system-tables-lazy-load 2020-05-31 18:06:08 +03:00
Denis Glazachev
e8144976fe Merge branch 'master' into ldap-per-user-authentication 2020-05-31 15:09:16 +04:00
Alexey Milovidov
25f941020b Remove namespace pollution 2020-05-31 00:57:37 +03:00
MovElb
142c63e487 done rebase 2020-05-30 23:02:11 +03:00
MovElb
b4b5c90343 squash 2020-05-30 20:05:05 +03:00
Alexey Milovidov
496fed5ed3 Remove "system_tables_lazy_load" option 2020-05-30 17:24:09 +03:00
Ivan Blinkov
d154415a5b adjust comments 2020-05-30 11:13:04 +03:00
Alexey Milovidov
b79020d0a8 Emit a warning if server was build in debug or with sanitizers 2020-05-30 00:41:33 +03:00
Ivan Blinkov
d033941399 try to fix merge issues 2020-05-29 23:20:28 +03:00
Ivan Blinkov
0386e526b2 grammar 2020-05-29 23:03:59 +03:00
Ivan Blinkov
95ee742ee7 merge master 2020-05-29 22:56:10 +03:00
Ivan Blinkov
4ef322274d Add integration test 2020-05-29 22:53:16 +03:00
Denis Glazachev
2863de750e Merge branch 'master' into ldap-per-user-authentication 2020-05-28 01:30:52 +04:00
Denis Glazachev
d74f1357d4 Add LDAP authentication support 2020-05-28 01:06:33 +04:00
Ivan Blinkov
709b4f42c8 Prototype sending crash reports on segfaults 2020-05-27 22:11:04 +03:00
Ivan
5b858de374
Split programs/server into actual program and library (#11186) 
* Split programs/server into actual program and library
 2020-05-27 20:52:52 +03:00
Alexey Milovidov
7e1813825b Return old names of macros 2020-05-24 01:24:01 +03:00
Alexey Milovidov
61cc605ee7 Remove old method 2020-05-24 01:21:29 +03:00
Alexey Milovidov
cd6d3c7509 Progress on task 2020-05-23 22:55:06 +03:00
Alexey Milovidov
f13ccfc8e8 Progress on task 2020-05-23 22:43:13 +03:00
Alexey Milovidov
7e2fb9ad65 Apply all transformations again 2020-05-23 22:38:30 +03:00
Alexey Milovidov
29762240de Remove duplicate whitespaces (preparation) 2020-05-23 22:31:54 +03:00
Alexey Milovidov
86b4d5a86f Remove duplicate whitespaces (preparation) 2020-05-23 22:09:06 +03:00
Alexey Milovidov
9d2a0d2dd7 Apply all transformations again 2020-05-23 21:59:49 +03:00
Alexey Milovidov
a2ad11897f Remove duplicate whitespaces (preparation) 2020-05-23 21:53:58 +03:00
Alexey Milovidov
1f13515a65 Make all LOG in single line (preparation) 2020-05-23 21:31:37 +03:00
Alexey Milovidov
3a09f9ca1c find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_[^\W_]+\([^,]+, [^<+)"]+\);' | xargs sed -i -r -e 's/(LOG_[^\W_]+)\(([^,]+), ([^<+)"]+)\);/\1_FORMATTED(\2, \3);/' 2020-05-23 20:29:56 +03:00
Alexey Milovidov
f69cbdcbfc find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" \+ [^+]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" \+ ([^+]+)\);/\1_FORMATTED(\2, "\3{}", \4);/' 2020-05-23 20:09:37 +03:00
Alexey Milovidov
533f86278a find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5{}\7", \4, \6);/' 2020-05-23 20:00:41 +03:00
Alexey Milovidov
8042e5febe find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}\5{}", \4, \6);/' 2020-05-23 19:58:15 +03:00
Alexey Milovidov
e391b77d81 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5", \4);/' 2020-05-23 19:56:05 +03:00
Alexey Milovidov
ee4ffbc332 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}", \4);/' 2020-05-23 19:47:56 +03:00
Alexey Milovidov
8d2e80a5e2 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+"\)' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+, "[^"]+")\)/\1_FORMATTED(\2)/' 2020-05-23 19:42:39 +03:00
Alexey Milovidov
1a67c89c26 Enable mlock by default 2020-05-22 14:31:27 +03:00
Nikolai Kochetov
f7456f8320
Merge pull request #11004 from ClickHouse/remove-experimental-use-processors-flag-2 
Remove allow_processors flag from executeQuery()
 2020-05-22 10:47:15 +03:00
alexey-milovidov
2d9433242e
Merge pull request #11097 from ClickHouse/azat-memory-tracking-2 
Merging #11022
 2020-05-21 20:19:59 +03:00
alexey-milovidov
765510d5c3
Merge pull request #10991 from ClickHouse/enable-clang-tidy-for-programs-and-utils 
Enable clang-tidy for programs and utils
 2020-05-21 15:12:07 +03:00
Azat Khuzhin
09474ebea5 Fix memory tracking for two-level GROUP BY when not all rows read from Aggregator (TCP) 
Example of such cases:
- SELECT GROUP BY LIMIT
- SELECT GROUP BY with subsequent MEMORY_LIMIT_EXCEEDED error

And it should be two-level aggregation, since otherwise there will be
only one hashtable which will be cleared correctly, only if you have
two-level GROUP BY some of hashtables will not be cleared since nobody
consume rows.

Before this patch:
    09:39.015292 [ 3070801 ] {609a0610-e377-4132-9cf3-f49454cf3c96} <Information> executeQuery: Read 1000000 rows, 7.63 MiB in 0.707 sec., 1413826 rows/sec., 10.79 MiB/sec.
    09:39.015348 [ 3070801 ] {609a0610-e377-4132-9cf3-f49454cf3c96} <Debug> MemoryTracker: Peak memory usage (for query): 51.93 MiB.
    09:39.015942 [ 3070801 ] {} <Trace> Aggregator: Destroying aggregate states <-- **problem**
    09:39.017057 [ 3070801 ] {} <Trace> Aggregator: Destroying aggregate states <--
    09:39.017961 [ 3070801 ] {} <Debug> MemoryTracker: Peak memory usage (for query): 51.93 MiB.
    09:39.018029 [ 3070801 ] {} <Information> TCPHandler: Processed in 0.711 sec.

After this patch:
    16:24.544030 [ 3087333 ] {79da208a-b3c0-48d4-9943-c974a3cbb6ea} <Information> executeQuery: Read 1000000 rows, 7.63 MiB in 0.599 sec., 1670199 rows/sec., 12.74 MiB/sec.
    16:24.544084 [ 3087333 ] {79da208a-b3c0-48d4-9943-c974a3cbb6ea} <Debug> MemoryTracker: Peak memory usage (for query): 72.11 MiB.
    16:24.544398 [ 3087333 ] {79da208a-b3c0-48d4-9943-c974a3cbb6ea} <Trace> Aggregator: Destroying aggregate states
    16:24.545485 [ 3087333 ] {79da208a-b3c0-48d4-9943-c974a3cbb6ea} <Trace> Aggregator: Destroying aggregate states
    16:24.547053 [ 3087333 ] {} <Debug> MemoryTracker: Peak memory usage (for query): 72.11 MiB.
    16:24.547093 [ 3087333 ] {} <Information> TCPHandler: Processed in 0.603 sec.
 2020-05-21 02:06:55 +03:00
Nikolai Kochetov
1183bad4d0 Fix build. 2020-05-20 22:01:36 +03:00
alexey-milovidov
07924f0ae1
Merge pull request #11029 from azat/system_tables_lazy_load 
Disable system_tables_lazy_load by default
 2020-05-20 21:15:09 +03:00
alesapin
a416813597
Merge pull request #10777 from NanoBjorn/refactor-reservations 
VolumePtr instead of DiskPtr in MergeTreeData*
 2020-05-19 13:27:11 +03:00
Azat Khuzhin
ef01eb954f Add note about system_tables_lazy_load into config.xml 2020-05-19 02:41:18 +03:00
Artem Zuikov
97c7447130
Bitonic sort improvements (#10934) 2020-05-19 00:41:23 +03:00
alexey-milovidov
551fbf8519
Update HTTPHandlerFactory.cpp 2020-05-18 22:46:32 +03:00
Artem Zuikov
bfcbc08bba arcadia sync fixes 2020-05-18 15:50:23 +03:00
Artem Zuikov
0fe723acc9
Try to fix arcadia sync (#10959) 2020-05-18 13:26:23 +03:00
alesapin
07cb21ccb7
Merge branch 'master' into refactor-reservations 2020-05-18 11:43:48 +03:00
Alexey Milovidov
f54435e7fd Fix clang-tidy 2020-05-18 11:08:55 +03:00
Alexey Milovidov
f4ac1eec04 Merge branch 'master' into quota-key-in-client 2020-05-17 08:15:13 +03:00
Gleb Novikov
1a25ac6e1f Merge branch 'master' into refactor-reservations 2020-05-16 23:34:45 +03:00
Nikolai Kochetov
caab379602
Merge pull request #10932 from ClickHouse/pulling-executor 
Add PullingPipelineExecutor.
 2020-05-16 23:09:18 +03:00
Nikolai Kochetov
6ac93e9a61 Try simplify code. 2020-05-15 22:20:02 +03:00
Nikolai Kochetov
83a35a7390 try fix tests. 2020-05-15 20:41:12 +03:00
alesapin
9ae37a054c Fix build without openCL 2020-05-15 11:31:32 +03:00
Artem Zuikov
8e8a2a17d6 build fixes 2020-05-15 03:01:31 +03:00
Ri
fc7afaa639
Bitonic sort on GPU (OpenCL) (#10232) 2020-05-15 03:01:14 +03:00
Nikolai Kochetov
681955661f Add PullingPipelineExecutor. 2020-05-15 00:03:38 +03:00
alexey-milovidov
ddc84163a7
Merge pull request #10486 from azat/dist-send-on-INSERT 
Fix distributed send that are scheduled by INSERT query
 2020-05-11 06:28:35 +03:00
Gleb Novikov
57b6fb6d80 Merge branch 'master' into refactor-reservations 2020-05-10 13:01:45 +03:00
Alexey Milovidov
894a2ccb4b Merge branch 'master' into even-more-warnings 2020-05-10 06:07:50 +03:00
Alexey Milovidov
9872964290 Enable "allow_introspection_functions" for development 2020-05-10 04:18:16 +03:00
Alexey Milovidov
4a13aba8de Fix build 2020-05-10 02:34:24 +03:00
Alexey Milovidov
a576a4fbcd Enable extra warnings for base, utils, programs 2020-05-10 01:59:34 +03:00
Gleb Novikov
390b39b272 VolumePtr instead of DiskPtr in MergeTreeData* 2020-05-10 00:24:15 +03:00
Ivan
85d783c247
Poco contrib refactoring (#10396) 
* Remove config_common.h
* Refactor libcpuid contrib
* Remove support for libcpuinfo
* Define USE_CPUID in Arcadia
* Refactor Poco libraries
 2020-05-08 17:11:19 +03:00
Alexander Tokmakov
d35150ecf3 fix http readonly request stucks on client close 2020-05-05 23:32:34 +03:00
alexey-milovidov
114f09cfca
Merge pull request #10666 from NanoBjorn/refactor-volume 
Volumes and storages refactoring
 2020-05-05 16:25:26 +03:00
Gleb Novikov
c637d99e07 Volumes and storages refactoring: 
1. Moved Volume to separate file
  2. Created IVolume interface and implemented current behaviour in implementation of new interface — VolumeJBOD
  3. Replaced all old volume usages with new VolumeJBOD. Where it is unnecessary to have JBOD — left just IVolume.
  4. Removed old Volume completely
  5. Moved StoragePolicy to separated files
  6. Moved DiskSelector to separated files
  7. Removed DiskSpaceMonitor file
 2020-05-04 23:15:38 +03:00
Alexey Milovidov
99c18c5a09 Check for suspicious codecs #4966 2020-05-04 03:11:49 +03:00
Alexey Milovidov
bb6eabd82d Fix HTTP response code for some parse errors #10636 2020-05-03 22:47:07 +03:00
Azat Khuzhin
8ccaf1bf32 Add internal_replication example config.xml 2020-05-03 14:46:45 +03:00
Alexey Milovidov
c40ffb88c6 Comments in config.xml; reset values for production use case 2020-05-01 22:18:35 +03:00
Alexey Milovidov
600b396f1b Merge remote-tracking branch 'origin/master' into sampling-memory-profiler 2020-05-01 21:48:20 +03:00
Alexander Kazakov
e9baaa439b
Implementation of new system metrics provider (Procfs) (#10544) 
* New metrics provider (Procfs) + Refactored TasksStatsCounters

* Trivial statless test that ProcFS is provided

* Trivial perf test for ProcfsMetricsProvider

Co-authored-by: alexey-milovidov <milovidov@yandex-team.ru>
 2020-05-01 21:47:41 +03:00
Alexey Milovidov
293ae88e7f Add sampling memory profiler 2020-04-30 16:25:17 +03:00
alexey-milovidov
cef5f12186
Merge pull request #10542 from azat/logger.levels.logger 
Ability to set logger level for loggers that have non allowed in xml tags symbols
 2020-04-29 00:13:52 +03:00
tavplubix
11c3493676
Merge pull request #10547 from ClickHouse/zhang2014/feature/ISSUES-5436 
Merging #7572
 2020-04-28 12:40:48 +03:00
Alexander Tokmakov
aeac8cb621 fix 2020-04-28 10:19:37 +03:00
Alexander Tokmakov
9d43c2f6c2 save compiled regex 2020-04-28 02:05:15 +03:00
Alexander Tokmakov
e72a484beb minor fixes 2020-04-28 02:05:15 +03:00
Azat Khuzhin
f767309574 Ability to set logger level for loggers that have non allowed in xml tags symbols 2020-04-27 21:30:29 +03:00
Azat Khuzhin
9ff0aaac9c Add example of logger.levels usage into config.xml 2020-04-27 21:21:01 +03:00
zhang2014
d56002b81f ISSUES-5436 fix ya make 2020-04-27 20:17:16 +08:00
zhang2014
81230944a4 ISSUES-5436 add integration test 2020-04-27 18:49:32 +08:00
Alexander Tokmakov
04d6b59ac0 Merge branch 'master' into database_atomic 2020-04-23 17:31:37 +03:00
zhang2014
0070f75218 ISSUES-5436 fix integration test failure & add test 2020-04-23 17:11:20 +08:00
Alexey Milovidov
07dcf40a6a Simple server wide memory profiler 2020-04-22 20:52:21 +03:00
Alexander Tokmakov
fefbbd37df Merge branch 'master' into database_atomic 2020-04-22 17:02:30 +03:00
Alexey Milovidov
0482e2e3ea Implement suggestion from Sergey Veletsky 2020-04-22 01:40:18 +03:00
alexey-milovidov
8c2839d3c9
Merge pull request #10308 from ClickHouse/shared-context-lifetime 
Shared context lifetime
 2020-04-21 00:12:45 +03:00
Alexander Tokmakov
b29bddac12 Merge branch 'master' into database_atomic 2020-04-20 14:09:09 +03:00
zhang2014
318ab3b51e ISSUES-5436 try fix build failure & pvs & style 2020-04-20 12:49:17 +08:00
Alexey Milovidov
1d885b3ff9 Merge branch 'master' into quota-key-in-client 2020-04-20 02:15:37 +03:00
Alexey Milovidov
2987e70aab Introduce "max_server_memory_usage" setting 2020-04-20 01:59:21 +03:00
Alexey Milovidov
ec6b176f3b Rework total memory tracker #10293 2020-04-20 01:59:21 +03:00
zhang2014
57cbecf935 ISSUES-5436 reworking predefine http 2020-04-20 03:18:36 +08:00
zhang2014
835dc4c445 After merge upsream master fix 2020-04-20 02:26:42 +08:00
zhang2014
b82eee5255 ISSUES-5436 fix review suggestions & add some '?' re2 test 2020-04-20 02:26:26 +08:00
zhang2014
8105a9bbe0 ISSUES-5436 fix review suggestions 2020-04-20 02:26:06 +08:00
tai
dafba9d66a ISSUES-5436 add integration test for custom http 2020-04-20 02:25:43 +08:00
zhang2014
847f7ab439 ISSUES-5436 fix build failure & fix test failure 2020-04-20 02:25:43 +08:00
zhang2014
6b716e511b ISSUES-5436 support custom http [part 6] 2020-04-20 02:25:12 +08:00
zhang2014
7aef95b082 ISSUES-5436 support custom http [part 5] 2020-04-20 02:23:18 +08:00
zhang2014
fd00757178 ISSUES-5436 support custom http [part 4] 2020-04-20 02:23:18 +08:00
zhang2014
60abff3341 ISSUES-5436 support custom http [part 3] 2020-04-20 02:23:18 +08:00
zhang2014
183eb82b7c ISSUES-5436 support custom http [part 2] 2020-04-20 02:23:18 +08:00
zhang2014
159ba24fca ISSUES-5436 support custom http 2020-04-20 02:23:18 +08:00
Nikolai Kochetov
84faa9af26 Merge branch 'master' into shared-context-lifetime 2020-04-17 19:13:13 +03:00
Nikolai Kochetov
ce11662ab1 Fix build. 2020-04-17 15:58:52 +03:00
Nikolai Kochetov
d51a9c551b Remove shared_ptr to SharedContext from Context. 2020-04-17 12:47:40 +03:00
alexey-milovidov
17e7d4d88a
Merge pull request #10307 from abyss7/arcadia-4 
Changes for auto-sync with Arcadia
 2020-04-17 05:08:34 +03:00
Nikolai Kochetov
025093d354 Fix build. 2020-04-16 17:51:33 +03:00
Nikolai Kochetov
d6db27e372 Modify context. 2020-04-16 17:37:38 +03:00
Ivan Lezhankin
e230632645 Changes required for auto-sync with Arcadia 2020-04-16 15:31:57 +03:00
alexey-milovidov
a92a6d8e63
Merge pull request #10274 from ClickHouse/avoid-error-message-on-rst 
Avoid error message on RST
 2020-04-15 17:33:35 +03:00
alexey-milovidov
2a48b0ecd7
Merge pull request #10184 from xzkostyan/feature-zookeeper-ssl 
Optional secured communication between ClickHouse and Zookeeper
 2020-04-15 07:09:14 +03:00
Alexey Milovidov
ddd840a81b Avoid error message on RST 2020-04-15 06:56:42 +03:00
Alexey Milovidov
fe59524481 Implemented in a different way 2020-04-15 04:59:14 +03:00
Alexey Milovidov
c22f91db01 Allow to pass quota_key in clickhouse-client 2020-04-15 04:15:05 +03:00
Alexander Tokmakov
6c0d9723e0 Merge branch 'master' into database_atomic 2020-04-13 18:47:47 +03:00
Alexander Tokmakov
5e6d4b9449 Merge branch 'master' into database_atomic 2020-04-12 16:35:44 +03:00
BohuTANG
e7f399f652 MySQLHandler: max_allowed_packet returned by default when server setup with select variables 2020-04-12 13:37:39 +08:00
Vitaly Baranov
ca634f603a Revert resetting the settings to their defaults after each query in TCPHandler. 2020-04-10 21:09:59 +03:00
Alexander Tokmakov
033d7f9b32 Merge branch 'master' into database_atomic 2020-04-10 17:12:09 +03:00
Vitaly Baranov
8a19f6d322
Merge pull request #10115 from azat/path.xml-access_control_path 
Rewrite access_control_path in path.xml
 2020-04-09 11:38:08 +03:00
Konstantin Lebedev
b666f60af8 Optional secured communication between ClickHouse and Zookeeper 2020-04-09 01:40:54 +03:00
BohuTANG
f48fdda678
Enhanced compatibility with native mysql-connector-java(JDBC) (#10021) 
* Skip the `/* comments */ SELECT @@variables ...` from mysql-connector-java setup for MySQL Handler #9336

mysql-connector setup query:
/* mysql-connector-java-5.1.38 ( Revision: ${revinfo.commit} ) */SELECT  @@session.auto_increment_increment AS auto_increment_increment, @@character_set_client AS character_set_client, @@character_set_connection AS character_set_connection, @@character_set_results AS character_set_results, @@character_set_server AS character_set_server, @@init_connect AS init_connect, @@interactive_timeout AS interactive_timeout...

ClickHouse side Error:
{} <Error> executeQuery: Code: 62, e.displayText() = DB::Exception: Syntax error: failed at position 74: @@session.auto_increment_increment AS auto_increment_increment, @@character_set_client AS character_set_client, @@character_set_connection AS character_set_conn. Expected one of: CAST, NULL...

Client side Exception:
java.sql.SQLException: Syntax error: failed at position 74: @@session.auto_increment_increment AS auto_increment_increment, @@character_set_client AS character_set_client, @@character_set_connection AS character_set_conn. Expected one of: CAST...

* add repalce 'SHOW VARIABLES' for mysql-connector-java-5.1.34 #9336

* Add java client(JDBC) integration test to test_mysql_protocol

* shift out java tests from dbms

* Update MySQLHandler.cpp

* Update MySQLHandler.cpp

* test_mysql_protocol: add Test.java exit code 1 when expection

Co-authored-by: alexey-milovidov <milovidov@yandex-team.ru>
 2020-04-09 00:52:19 +03:00
Alexander Tokmakov
dd1590830b Merge branch 'master' into database_atomic 2020-04-08 22:00:46 +03:00
Azat Khuzhin
ed25ac8b12 Rewrite access_control_path in path.xml 
Follow-up for: #9811
 2020-04-08 10:45:14 +03:00
Vitaly Baranov
bc3e2cb48c Add the option "access_control_path" to server config. 2020-04-07 23:31:59 +03:00
Vitaly Baranov
b71ff6160a Add the "access_management" option to users.xml which enables a user to manage access rights. 
The option is turned off by default.
 2020-04-07 23:31:54 +03:00
Alexander Tokmakov
9c67d2716e Merge branch 'master' into database_atomic 2020-04-03 03:19:11 +03:00
Ivan
97f2a2213e
Move all folders inside /dbms one level up (#9974) 
* Move some code outside dbms/src folder
* Fix paths
 2020-04-02 02:51:21 +03:00