thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-19 14:11:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
101,476 Commits 9,138 Branches 1,778 Tags 2 GiB
3239516521
Commit Graph

3026 Commits

Author SHA1 Message Date
Sema Checherinda
e436b4f4cc Metric for the number of detached parts 2022-08-30 02:12:57 +02:00
Alexander Tokmakov
1c6dea52e0
Update config.xml 2022-08-29 15:50:05 +03:00
Alexey Milovidov
84f9432e7e
Merge pull request #40724 from azat/update-bash-completion 
Update available formats for bash completion
 2022-08-29 00:14:48 +03:00
Alexey Milovidov
f462160bd1 Merge remote-tracking branch 'origin/master' into remove-useless-line 2022-08-28 21:00:29 +02:00
Alexey Milovidov
2fa8d212e6 Fix very strange test 2022-08-28 21:00:15 +02:00
Alexey Milovidov
deac6cfb5d Minor modifications 2022-08-28 20:49:39 +02:00
Azat Khuzhin
29877d3992 Update available formats for bash completion 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-28 17:22:32 +02:00
Alexey Milovidov
ddff5e9145
Merge pull request #40712 from ClickHouse/ci-cmake-self-extractable 
cmake: Disable default ENABLE_CLICKHOUSE_SELF_EXTRACTING
 2022-08-28 17:12:19 +03:00
Alexey Milovidov
b42d26acfe Remove one line from XML, because we do not care 2022-08-28 02:44:02 +02:00
Alexey Milovidov
632867dbff Fix error 2022-08-28 02:40:43 +02:00
Alexey Milovidov
3139d4a79a Fix error 2022-08-28 02:38:46 +02:00
Yakov Olkhovskiy
31a7ed09a1 disable default ENABLE_CLICKHOUSE_SELF_EXTRACTING and add to env 2022-08-27 21:08:01 +00:00
Alexey Milovidov
b14bd43590 Fix error 2022-08-27 20:25:06 +02:00
Alexey Milovidov
b70373d429 Fix error 2022-08-27 20:23:15 +02:00
Alexey Milovidov
6414a01987 Rewrite NLP tests from integration to functional 2022-08-26 22:36:34 +02:00
Alexey Milovidov
77e15a24d0 Add missing tests for legacy geobase 2022-08-26 22:17:49 +02:00
Alexey Milovidov
34b5d267f3
Merge pull request #40461 from ClickHouse/dashboard 
Dashboard
 2022-08-23 15:34:28 +03:00
Alexey Milovidov
0252695c6e
Merge pull request #40224 from ClickHouse/alexey-milovidov-patch-4 
Check what will be if I enable `concurrency-mt-unsafe` in `clang-tidy`
 2022-08-22 23:12:01 +03:00
Alexey Milovidov
6444afac52 Merge branch 'master' into dashboard 2022-08-22 01:30:21 +02:00
Robert Schulze
e8b3f56733
Limit suppression to a specific warning 2022-08-21 18:24:17 +00:00
Alexey Milovidov
000f879779 Add comments 2022-08-21 13:31:04 +02:00
Alexey Milovidov
861e5dbb76 Add comments 2022-08-21 13:26:32 +02:00
Alexey Milovidov
ab079b270a Add comments 2022-08-21 13:07:46 +02:00
Alexey Milovidov
fa9df87198 Fix margin 2022-08-21 12:56:23 +02:00
Alexey Milovidov
458d21dd3c Add "readpassphrase" as a dependency for clickhouse-local 2022-08-21 12:12:11 +02:00
Alexey Milovidov
9635fa817b Embedded dashboard 2022-08-21 12:11:25 +02:00
Alexey Milovidov
4e0bf0dd34 Add a file 2022-08-21 12:11:04 +02:00
Alexey Milovidov
3649aac593 Fix style 2022-08-20 17:59:51 +02:00
Alexey Milovidov
74e1f4dc61 Fix clang-tidy 2022-08-20 17:09:20 +02:00
Sergei Trifonov
7bbbe0a4db fix darwin build 2022-08-18 19:31:25 +02:00
Sergei Trifonov
b064021cec fix build 2022-08-18 16:57:31 +02:00
Sergei Trifonov
42a711015f split concurrent_threads_soft_limit setting into two: concurrent_threads_soft_limit_{num,ratio_to_cores} 2022-08-18 14:51:08 +02:00
Nikita Mikhaylov
a7c3f23dfb
Merge branch 'master' into auto_convert_ordinary_to_atomic 2022-08-17 00:38:25 +02:00
Alexander Tokmakov
c9bb91c31f
Merge pull request #40232 from ClickHouse/fix_a_bug_with_symlinks_detection 
Fix a bug with symlinks detection
 2022-08-15 20:28:30 +03:00
Alexander Tokmakov
6f5a7c3bf7 fix a bug with symlinks detection 2022-08-15 12:30:47 +02:00
Alexey Milovidov
bada9ebefa
Merge branch 'master' into disable-zero-copy-replication 2022-08-14 10:40:16 +03:00
Alexey Milovidov
e774d28c43 Fix style 2022-08-14 04:16:48 +02:00
Alexey Milovidov
53ce2986de Display server-side time in clickhouse-benchmark by default 2022-08-14 03:33:42 +02:00
Alexey Milovidov
f0c8998471 Add warning message 2022-08-14 01:49:00 +02:00
Alexey Milovidov
42c358aa3c Add warning message 2022-08-14 01:47:36 +02:00
Kseniia Sumarokova
a6cfc7bc3b
Merge pull request #34651 from alexX512/master 
New caching strategies
 2022-08-12 17:23:37 +02:00
Nikita Taranov
17956cb668
Extend protocol with query parameters (#39906) 2022-08-12 14:28:35 +02:00
Anton Popov
0383250383 execute DROP queries for fuzzed tables 2022-08-12 00:32:21 +00:00
Nikolay Degterinsky
644be9dc42 Remove unneded ReadBuffers 2022-08-11 13:27:09 +00:00
Nikolay Degterinsky
1b8ca90fcc Add schema inference to clickhouse-obfuscator 2022-08-11 11:32:32 +00:00
Anton Popov
e347fa2f45 add fuzzer for table definition 2022-08-11 02:34:10 +00:00
Nikolay Degterinsky
496568101f
Merge pull request #39277 from ClibMouse/Structured-Logging-Support 
Structured logging support
 2022-08-10 22:57:48 +02:00
Kseniia Sumarokova
a4d513657c
Merge pull request #40010 from ClickHouse/catboost-further-preparation 
Further preparation for catboost integration into library-bridge
 2022-08-10 17:35:12 +02:00
Vladimir C
eb1ac41ac5
Merge pull request #40053 from ClickHouse/play-tab-in-textarea 
Play: recognize tab in textarea
 2022-08-10 12:53:28 +02:00
Robert Schulze
810221baf2
Assume unversioned server has version=0 and use tryParse() instead of from_chars() 2022-08-10 07:39:32 +00:00
Alexey Milovidov
34c8d2c3b4 Play: recognize tab in textarea 2022-08-10 08:33:58 +02:00
Mallik Hassan
666fb8dc42
Merge branch 'ClickHouse:master' into Structured-Logging-Support 2022-08-09 15:39:00 -03:00
Robert Schulze
e0d5020a92
Add simple versioning to the *-bridge-to-server protocol 
- In general, it is expected that clickhouse-*-bridges and
  clickhouse-server were build from the same source version (e.g. are
  upgraded "atomically"). If that is not the case, we should at least
  be able to detect the mismatch and abort.

- This commit adds a URL parameter "version", defined in a header shared
  by the server and bridges. The bridge returns an error in case of
  mismatch.

- The version is *not* send and checked for "ping" requests (used for
  handshake), only for regular requests send after handshake. This is
  because the internally thrown server-side exception due to HTTP
  failure does not propagate the exact HTTP error (it only stores the
  error as text), and as a result, the server-side handshake code
  simply retries in case of error with exponential backoff and finally
  fails with a "timeout error". This is reasonable as pings typically
  fail due to time out. However, without a rework of HTTP exceptions,
  version mismatch during ping would also appear as "timeout" which is
  too misleading. The behavior may be changed later if needed.

- Note that introducing a version parameter does not represent a
  protocol upgrade itself. Bridges older than the server will simply
  ignore the field. Only servers older than the bridges receive an error
  but such a situation should never occur in practice.
 2022-08-08 19:40:37 +00:00
Robert Schulze
9952ab1099
Prefix class names "LibraryBridge*Handler" with "ExternalDictionary" 
- necessary to disambiguate the names from "CatBoost"-"LibraryBridgeHandler"
  which will be added in a next step
 2022-08-08 17:16:46 +00:00
Yakov Olkhovskiy
d39e9f65de
Merge branch 'master' into fix-quota-key 2022-08-08 11:54:21 -04:00
Robert Schulze
ad0d060dc1
Merge pull request #39904 from ClickHouse/library-bridge-refactoring 
Prepare library-bridge for catboost integration
 2022-08-08 12:15:01 +02:00
Alexey Milovidov
64a1b0f2b8 Play UI: row numbers; cell selection; hysteresis 2022-08-08 04:25:03 +02:00
Alexey Milovidov
653b86d1e6 Play UI: row numbers; cell selection; hysteresis 2022-08-08 04:16:50 +02:00
Alexey Milovidov
2d83635643 Play UI: row numbers; cell selection; hysteresis 2022-08-08 03:57:02 +02:00
Alexey Milovidov
d914ddc766 Fix Play UI 2022-08-08 02:36:50 +02:00
alexX512
2be72560f4 Add loading of mark cache policy 2022-08-07 19:59:39 +00:00
alexX512
1133e42367 Add CacheBase clas instead of LRUCache and SLRUCache fo simpler configuration oache policiesf 2022-08-07 19:16:40 +00:00
Yakov Olkhovskiy
29622746ea
Merge pull request #39936 from ClickHouse/cmake-fix-self-extracting-split 
Don't create self-extracting clickhouse for split build
 2022-08-07 07:23:24 -04:00
Yakov Olkhovskiy
b1f45fa787 Don't create self-extracting clickhouse for split build 2022-08-05 21:48:40 -04:00
Alexey Milovidov
ac93f0a2ca
Merge pull request #39910 from santrancisco/enhance-playui-authentication 
Move username and password from URL parameters to Basic Authentication
 2022-08-06 01:58:09 +03:00
Alexander Tokmakov
f2c4cad255 add flag that enables automatic canversion from Ordinary to Atomic 2022-08-05 21:41:25 +02:00
santrancisco
c7f0598a6f Fall back to url parameters if file is opened locally 2022-08-05 13:10:45 +10:00
santrancisco
a25b858097 Move username and password from URL parameter to Basic Authentication header 2022-08-05 09:51:12 +10:00
Robert Schulze
20bb8a248e
Prepare server-side BridgeHelper for catboost integration 
Wall of text, sorry, but I also had to document some stuff for myself:

There are three ways to communicate data using HTTP:
- the HTTP verb: for our purposes, PUT and GET,
- the HTTP path: '/ping', '/request' etc.,
- the HTTP URL parameter(s), e.g. 'method=libNew&dictionary_id=1234'

The bridge will use different handlers for communication with the
external dictionary library and for communication with the catboost
library. Handlers are created based on a combination of the HTTP verb
and the HTTP method. More specifically, there will be combinations
- GET + '/extdict_ping'
- PUT + '/extdict_request'
- GET + '/catboost_ping'
- PUT + '/catboost_request'.
For each combination, the bridge expects a certain set of URL
parameters, e.g. for the first combination parameter "dictionary_id" is
expected.

Starting with this commit, the library-bridge creates handlers based on
the first two combinations (the latter two combinations will be added
later). This makes the handler creation mechanism consistent with it's
counterpart in xdbc-bridge.

For that, it was necessary to make both IBridgeHelper methods
"getMainURI()" and "getPingURI()" pure virtual so that derived classes
(LibraryBridgeHelper and XDBCBridgeHelper) must provide custom URLs with
custom paths.

Side note 1: Previously, LibraryBridgeHelper sent HTTP URL parameter
"method=ping" during handshake (PING) but the library-bridge ignored
that parameter. We now omit this parameter, i.e.
LibraryBridgeHelper::PING was removed. Again, this makes things
consistent with xdbc-bridge.

Side note 2: xdbc-bridge is unchanged in this commit. Therefore,
XDBCBridgeHelper now uses the HTTP paths previously in the base class.
For funny reason, XDBCBridgeHelper did not use
IBridgeHelper::getMainURI() - it generates the URLs by itself. I kept it
that way for now but provided an implementation of getMainURI() anyways.
 2022-08-04 19:29:51 +00:00
Robert Schulze
ea73b98fb9
Prepare library-bridge for catboost integration 
- Rename generic file and identifier names in library-bridge to
  something more dictionary-specific. This is needed because later on,
  catboost will be integrated into library-bridge.

- Also: Some smaller fixes like typos and un-inlining non-performance
  critical code.

- The logic remains unchanged in this commit.
 2022-08-04 19:26:51 +00:00
Yakov Olkhovskiy
23037daf17
Merge branch 'master' into fix-quota-key 2022-08-04 12:14:49 -04:00
Alexey Milovidov
834cbbedce
Merge branch 'master' into escape_diag_creds 2022-08-04 01:59:50 +03:00
Yakov Olkhovskiy
2e34b384c1 update tcp protocol, add quota_key 2022-08-03 15:44:08 -04:00
Mallik Hassan
87bde54ee5
Merge branch 'ClickHouse:master' into Structured-Logging-Support 2022-08-02 10:44:05 -03:00
Alexey Milovidov
5f62863ab9
Merge branch 'master' into escape_diag_creds 2022-08-02 05:50:01 +03:00
Heena Bansal
d8db482b2e
Merge branch 'master' into Issue_39395 2022-08-01 12:22:16 -04:00
Robert Schulze
bf574b9154
Merge pull request #39760 from ClickHouse/bit-fiddling 
Use std::popcount, ::countl_zero, ::countr_zero functions
 2022-08-01 17:04:51 +02:00
HeenaBansal2009
50c98789b8 Updated as per comments 2022-08-01 07:05:50 -07:00
HeenaBansal2009
800ed546be Updated as per comments 2022-08-01 07:03:36 -07:00
Ilya Yatsishin
c882bdc88e
Merge pull request #35968 from ClickHouse/interserver_listen_port 2022-08-01 13:57:17 +02:00
Alexander Tokmakov
80f9ba9186
Merge pull request #39690 from ClickHouse/show-addresses-in-stack-traces 
Configuration parameter to hide addresses in stack traces
 2022-08-01 14:20:37 +03:00
Alexey Milovidov
6a2f7d0c8f
Merge branch 'master' into interserver_listen_port 2022-07-31 22:51:38 +03:00
Alexey Milovidov
ebc4974188
Merge branch 'master' into escape_diag_creds 2022-07-31 22:46:10 +03:00
Robert Schulze
a7734672b9
Use std::popcount, ::countl_zero, ::countr_zero functions 
- Introduced with the C++20 <bit> header

- The problem with __builtin_c(l|t)z() is that 0 as input has an
  undefined result (*) and the code did not always check. The std::
  versions do not have this issue.

- In some cases, we continue to use buildin_c(l|t)z(), (e.g. in
  src/Common/BitHelpers.h) because the std:: versions only accept
  unsigned inputs (and they also check that) and the casting would be
  ugly.

(*) https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html
 2022-07-31 15:16:51 +00:00
Robert Schulze
dcc8751685
Disable harmful env var check to workaround failure to start the server 2022-07-31 08:55:07 +00:00
Robert Schulze
7c23e48b5b
Revert exclusion of libharmful (did not work anyways) 2022-07-31 08:05:12 +00:00
Robert Schulze
7fe106a0fb
Try to fix libharmful fail 2022-07-31 07:44:25 +00:00
Alexey Milovidov
fa9c3dcc48
Update programs/local/LocalServer.cpp 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2022-07-31 03:02:27 +03:00
Alexey Milovidov
a30dbed6b8
Update programs/local/LocalServer.cpp 
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
 2022-07-31 03:02:20 +03:00
Alexey Milovidov
f80a4c184e
Merge branch 'master' into interserver_listen_port 2022-07-31 01:22:32 +03:00
Alexey Milovidov
9adfd259d0 Support in clickhouse-local 2022-07-30 01:50:45 +02:00
Alexey Milovidov
c0d7b6efc3 Suggestions from @tavplubix 2022-07-30 01:45:06 +02:00
Alexey Milovidov
4aa96767d5 Merge branch 'master' of github.com:ClickHouse/ClickHouse into show-addresses-in-stack-traces 2022-07-30 01:44:04 +02:00
Robert Schulze
3d1797f75f
Merge remote-tracking branch 'origin/master' into no-split-binary 2022-07-29 12:17:43 +00:00
Dale Mcdiarmid
d9c585ecf1 Escape credentials for diagnostics tool 2022-07-29 12:25:03 +01:00
Azat Khuzhin
b90152b6ec Fix clickhouse-su building in splitted build 
- Add status log message
- Add it to clickhouse-bundle in shared build
- Move clickhouse-su.cpp into su.cpp, since executable does not have
  include directories of linked libraries (dbms here), only
  clickhouse-lib-su does, hence it cannot find includes

CI: https://github.com/ClickHouse/ClickHouse/runs/7566319416?check_suite_focus=true
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-29 11:36:51 +03:00
root
39369be318 commented out formatting tag in config.xml 2022-07-28 23:56:26 -07:00
root
f9e4d6370d modified integration test as suggested 2022-07-28 23:51:59 -07:00
Alexey Milovidov
552b517e59 Allow to hide addresses in stack traces 2022-07-29 00:13:02 +02:00
root
bec7408a0c reflected change requests asked on July 27 2022-07-28 09:20:28 -07:00
Robert Schulze
199e254777
Merge remote-tracking branch 'origin/master' into no-split-binary 2022-07-28 15:54:22 +00:00
Sergei Trifonov
4588e146a6
Merge pull request #37285 from arenadata/ADQM-445 
Add concurrent_threads_soft_limit parameter
 2022-07-28 11:46:43 +02:00
Roman Vasin
f35349eb08 Rename total_max_threads to concurrent_threads_soft_limit 2022-07-28 07:46:20 +00:00
Roman Vasin
37845389c8 Merge branch 'concurrency-control' of github.com:ClickHouse/ClickHouse into ADQM-445 2022-07-27 13:31:22 +00:00
Antonio Andelic
e62526720f Address PR comments 2022-07-27 07:51:30 +00:00
Antonio Andelic
35b61cc94e Merge branch 'master' into keeper-version-check 2022-07-26 07:47:57 +00:00
Alexey Milovidov
75d0232265
Merge pull request #39541 from ClickHouse/obfuscator-save-load 
Add save/load capabilities to Obfuscator
 2022-07-25 21:18:28 +03:00
Mallik Hassan
b45e44b7e7
Merge branch 'ClickHouse:master' into Structured-Logging-Support 2022-07-25 12:10:41 -03:00
Yakov Olkhovskiy
d0f5dcad25
Merge pull request #38936 from ClickHouse/cmake-self-extracting-executable 
Build self-extracting-executable utils
 2022-07-25 08:13:25 -04:00
Alexey Milovidov
1f4e7ea34c Add a test 2022-07-25 05:37:07 +02:00
Alexey Milovidov
dde89c9606 Fix errors 2022-07-25 03:56:32 +02:00
Alexey Milovidov
68f5a397d7 Fix errors 2022-07-25 03:49:09 +02:00
Alexey Milovidov
52ad48d1ba Better documentation 2022-07-25 03:32:01 +02:00
Alexey Milovidov
c09413e3b9 Add save/load capabilities to Obfuscator 2022-07-25 03:27:10 +02:00
Alexey Milovidov
071374b152 Remove SPLIT_BINARY 2022-07-24 01:15:54 +02:00
Antonio Andelic
afb6cb6824 Add KeeperContext 2022-07-23 14:45:31 +00:00
Antonio Andelic
3040ff0959 Merge branch 'master' into keeper-version-check 2022-07-22 08:07:55 +00:00
jasperzhu
614f3b14a2
Merge branch 'master' into dev_intel_iaa_deflate 2022-07-22 11:05:44 +08:00
HeenaBansal2009
06e8b78efa Added FT testcase 2022-07-21 19:46:36 -07:00
Kruglov Pavel
49c839bdc1
Merge pull request #39413 from vitlibar/fix-interactive-client-with-older-server 
Fix interactive client with older server
 2022-07-21 11:37:20 +02:00
Antonio Andelic
7d30ab80c4 Fix compatibility 2022-07-21 09:31:06 +00:00
Vitaly Baranov
da0bdd474b Fix code style. 2022-07-21 10:48:53 +02:00
Yakov Olkhovskiy
e7438cc433
Merge branch 'master' into cmake-self-extracting-executable 2022-07-21 00:25:08 -04:00
Yakov Olkhovskiy
b766552481 add native build for cross-compilation 2022-07-20 23:09:05 -04:00
Nikolai Kochetov
91043351aa Fixing build. 2022-07-20 20:30:16 +00:00
Vitaly Baranov
8873d16afe Fix interactive client with older server after https://github.com/ClickHouse/ClickHouse/pull/38970 2022-07-20 11:51:55 +02:00
HeenaBansal2009
d67e061915 Clickhouse-local fixes 2022-07-19 21:38:36 -07:00
jasperzhu
c044c67745
Merge branch 'master' into dev_intel_iaa_deflate 2022-07-19 15:32:57 +08:00
Azat Khuzhin
4f41d21626 Fix leaking of logger in clickhouse-disks 
CI found [1]:

    Direct leak of 256 byte(s) in 1 object(s) allocated from:
        0 0xd8cb88d in operator new(unsigned long) (/usr/bin/clickhouse+0xd8cb88d) (BuildId: 7a3fd7b485701220)
        1 0xde8943e in DB::DisksApp::main() build_docker/../programs/disks/DisksApp.cpp:157:41
        2 0x38dca887 in Poco::Util::Application::run() build_docker/../contrib/poco/Util/src/Application.cpp:334:8
        3 0xde8d72c in mainEntryClickHouseDisks(int, char**) build_docker/../programs/disks/DisksApp.cpp:219:20
        4 0xd8cf47f in main build_docker/../programs/main.cpp:445:12
        5 0x7f060ddce082 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x24082) (BuildId: 1878e6b475720c7c51969e69ab2d276fae6d1dee)

  CI: https://s3.amazonaws.com/clickhouse-test-reports/39299/37b4b52c12698e711aa931f10aec3909bca287b6/integration_tests__asan__actions__[2/3].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-18 12:35:53 +03:00
jinjunzh
dbac3a35b2 fixed cosmetic issues 2022-07-15 17:23:37 -04:00
root
af718f7e04 updated config.xml 2022-07-15 13:06:39 -07:00
root
03194eaeb9 Feature - Structured Logging Support resubmit 2022-07-15 10:01:30 -07:00
Vitaly Baranov
6bf7bffbeb Correct the list of always accessible system tables. 2022-07-15 15:44:29 +02:00
Vitaly Baranov
fbb2e14d54 Add new table function viewIfPermitted(). 2022-07-15 15:44:29 +02:00
Vitaly Baranov
c85b2b5732 Add option enabling that SELECT from the system database requires grant. 2022-07-15 15:44:25 +02:00
Robert Schulze
62149111d3
Fix build 2022-07-14 14:36:31 +00:00
Yakov Olkhovskiy
e5f165d909
Merge branch 'master' into cmake-self-extracting-executable 2022-07-13 16:09:18 -04:00
Robert Schulze
1a7727a254
Prefix overridden add_executable() command with "clickhouse_" 
A simple HelloWorld program with zero includes except iostream triggers
a build of ca. 2000 source files. The reason is that ClickHouse's
top-level CMakeLists.txt overrides "add_executable()" to link all
binaries against "clickhouse_new_delete". This links against
"clickhouse_common_io", which in turn has lots of 3rd party library
dependencies ... Without linking "clickhouse_new_delete", the number of
compiled files for "HelloWorld" goes down to ca. 70.

As an example, the self-extracting-executable needs none of its current
dependencies but other programs may also benefit.

In order to restore access to the original "add_executable()", the
overriding version is now prefixed. There is precedence for a
"clickhouse_" prefix (as opposed to "ch_"), for example
"clickhouse_split_debug_symbols". In general prefixing makes sense also
because overriding CMake commands relies on undocumented behavior and is
considered not-so-great practice (*).

(*) https://crascit.com/2018/09/14/do-not-redefine-cmake-commands/
 2022-07-11 19:36:18 +02:00
Antonio Andelic
37f799550b
Merge pull request #38072 from lingpeng0314/master 
Add Keeper related monitoring data
 2022-07-10 09:39:57 +02:00
jinjunzh
816e974ca2 rename deflate to deflate_qpl 2022-07-09 14:42:01 -04:00
jinjunzh
825b7511e4 improve cosmetics 2022-07-09 14:13:12 -04:00
Nikita Mikhaylov
a44157d81d
Allow globs in keys for clickhouse-extract-from-config tool (#38966) 2022-07-08 16:13:32 +02:00
jinjunzh
6fa036f6ee remove qpl from clickhouse-keeper 2022-07-06 23:48:09 -04:00
Yakov Olkhovskiy
8a3f124982 add self-extracting to clickhouse-bundle 2022-07-06 22:01:21 -04:00
Yakov Olkhovskiy
c6db15458a build utils/self-extracting-executable/compressor whenever we want to build compressed binary 2022-07-06 20:40:41 -04:00
jinjunzh
ba38969e9c enable qpl for clickhouse-keeper build 2022-07-05 15:28:41 -04:00
Alexey Milovidov
1f2266e45f
Merge pull request #38619 from ClickHouse/filimonov-clickhouse-su 
Fix exception messages in clickhouse su
 2022-07-05 07:11:20 +03:00
Mikhail f. Shiryaev
7a58a8c19b
Merge pull request #38647 from ClickHouse/clickhouse-diagnostics 
Clickhouse diagnostics
 2022-07-04 12:14:33 +02:00
Mikhail f. Shiryaev
446ead1af4
Fix typos issues it clickhouse-diagnostics 2022-07-04 10:59:39 +02:00
Mikhail f. Shiryaev
664d43e348
Add disabling tag no_docker to some go tests 2022-07-04 10:59:39 +02:00
lingpeng0314
cca94f01ae add function tryGetKeeperDispatcher for context 
- to make sure other case will get error if dispatcher not initialized
 2022-07-04 13:35:08 +08:00
Mikhail f. Shiryaev
3513b41683
Fix linter issue 2022-07-01 11:43:14 +02:00
Mikhail f. Shiryaev
1553f67809
Can set version in Makefile 2022-07-01 11:43:13 +02:00
Mikhail f. Shiryaev
a70cda9670
Create a path for clickhouse-diagnostics installation 2022-07-01 11:43:12 +02:00
Mikhail f. Shiryaev
51556704e0
Rename module to reflect the current state 2022-07-01 11:43:11 +02:00
Mikhail f. Shiryaev
9531bbf497
Move clickhouse-diagnostics to programs 2022-07-01 11:43:07 +02:00
Robert Schulze
59236d60c9
Merge pull request #38654 from ClickHouse/better-naming-for-split-debug-symbols 
Better naming for stuff related to splitted debug symbols
 2022-07-01 09:28:41 +02:00
lingpeng0314
03a5ce542d Remove Exception in getKeeperDispatcher 
- to handle the nil case by caller
 2022-07-01 11:27:40 +08:00
Robert Schulze
bb358617e1
Better naming for stuff related to splitted debug symbols 
The previous name was slightly misleading, e.g. it is not about
"intalling stripped binaries" but about splitting debug symbols from the
binary.
 2022-06-30 23:41:27 +02:00
Yakov Olkhovskiy
dd132d1beb
Merge pull request #38447 from ClickHouse/utility-self-extracting 
Compress clickhouse executable
 2022-06-30 13:18:40 -04:00
Nikita Mikhaylov
e66114984c
Update programs/su/clickhouse-su.cpp 2022-06-30 12:44:59 +02:00
Nikita Mikhaylov
b2cc6787ee
Update programs/su/clickhouse-su.cpp 2022-06-30 12:44:30 +02:00
filimonov
3949480ed3
Fix exception messages in clickhouse su 
Fix exception messages in
 2022-06-30 11:38:05 +02:00
Yakov Olkhovskiy
78ea290789 add remove command 2022-06-29 06:47:21 -04:00
jinjunzh
fe451a4317 remove compress sync interface 2022-06-28 22:52:11 -04:00
Yakov Olkhovskiy
5d36994c4d self-extracting requires utils (uses utils/self-extracting-executable/compressor) 2022-06-27 11:41:23 -04:00
Alexander Tokmakov
7c37079a96 Merge branch 'master' into deprecate_ordinary_database 2022-06-27 14:46:59 +02:00
mergify[bot]
4e5fd226c8
Merge branch 'master' into utility-self-extracting 2022-06-27 12:26:16 +00:00
Yakov Olkhovskiy
8ce6b8226d
Update CMakeLists.txt 2022-06-27 08:25:21 -04:00
Yakov Olkhovskiy
39ea5ffdcb compress clickhouse executable, new target 'self-extracted' is added 2022-06-27 01:36:27 -04:00
jasperzhu
b2f98fa73b
Merge branch 'master' into dev_intel_iaa_deflate 2022-06-27 10:21:21 +08:00
Alexey Milovidov
b89f01438e Lower mutex scope 2022-06-25 03:05:55 +02:00
Kseniia Sumarokova
e1823f4c0a
Merge pull request #38364 from kssenii/better-error-message-in-dbc 
Better error message for failed odbc query
 2022-06-24 18:30:54 +02:00
Alexander Tokmakov
31dcc7634e Merge branch 'master' into deprecate_ordinary_database 2022-06-24 18:16:07 +02:00
kssenii
dc73042d62 Better error messafe 2022-06-24 01:05:33 +02:00
Alexander Gololobov
50438a4b05 Fixed uninitialized optional 2022-06-23 23:55:21 +02:00
Alexander Tokmakov
dbf2763788 automatically convert system database to Atomic 2022-06-23 21:38:43 +02:00
Alexander Gololobov
02f3921c69 Added --recursive to clickhouse-disks list 2022-06-23 20:39:20 +02:00
Kseniia Sumarokova
b844c36a9d
Merge pull request #38150 from kssenii/clickhouse-disks-fixes 
Some fixes for clickhouse-disks
 2022-06-23 10:21:44 +02:00
Kseniia Sumarokova
3f83d2f8b3
Update ICommand.cpp 2022-06-22 13:00:48 +02:00
Alexey Milovidov
ba05b3edab
Merge pull request #38197 from ClickHouse/play-charts 
Add primitive charting capabilities
 2022-06-22 06:42:05 +03:00
Robert Schulze
0d80874d40
Merge pull request #38068 from ClickHouse/clang-tsa 
Support for Clang Thread Safety Analysis (TSA)
 2022-06-21 20:19:33 +02:00
kssenii
47a118beab More fixes 2022-06-21 16:40:22 +02:00
mergify[bot]
5b24939369
Merge branch 'master' into play-charts 2022-06-21 14:20:52 +00:00
Roman Vasin
3de3ae1b64 Set ConcurrencyControl::Unlimited when total_max_threads 0 or undefined 2022-06-21 16:39:53 +03:00
Alexey Milovidov
7e9a3796e9 Fix wrong docs 2022-06-21 00:09:55 +02:00
Robert Schulze
55b39e709d
Merge remote-tracking branch 'origin/master' into clang-tsa 2022-06-20 16:39:32 +02:00
Robert Schulze
5a4f21c50f
Support for Clang Thread Safety Analysis (TSA) 
- TSA is a static analyzer build by Google which finds race conditions
  and deadlocks at compile time.

- It works by associating a shared member variable with a
  synchronization primitive that protects it. The compiler can then
  check at each access if proper locking happened before. A good
  introduction are [0] and [1].

- TSA requires some help by the programmer via annotations. Luckily,
  LLVM's libcxx already has annotations for std::mutex, std::lock_guard,
  std::shared_mutex and std::scoped_lock. This commit enables them
  (--> contrib/libcxx-cmake/CMakeLists.txt).

- Further, this commit adds convenience macros for the low-level
  annotations for use in ClickHouse (--> base/defines.h). For
  demonstration, they are leveraged in a few places.

- As we compile with "-Wall -Wextra -Weverything", the required compiler
  flag "-Wthread-safety-analysis" was already enabled. Negative checks
  are an experimental feature of TSA and disabled
  (--> cmake/warnings.cmake). Compile times did not increase noticeably.

- TSA is used in a few places with simple locking. I tried TSA also
  where locking is more complex. The problem was usually that it is
  unclear which data is protected by which lock :-(. But there was
  definitely some weird code where locking looked broken. So there is
  some potential to find bugs.

*** Limitations of TSA besides the ones listed in [1]:

- The programmer needs to know which lock protects which piece of shared
  data. This is not always easy for large classes.

- Two synchronization primitives used in ClickHouse are not annotated in
  libcxx:
  (1) std::unique_lock: A releaseable lock handle often together with
      std::condition_variable, e.g. in solve producer-consumer problems.
  (2) std::recursive_mutex: A re-entrant mutex variant. Its usage can be
      considered a design flaw + typically it is slower than a standard
      mutex. In this commit, one std::recursive_mutex was converted to
      std::mutex and annotated with TSA.

- For free-standing functions (e.g. helper functions) which are passed
  shared data members, it can be tricky to specify the associated lock.
  This is because the annotations use the normal C++ rules for symbol
  resolution.

[0] https://clang.llvm.org/docs/ThreadSafetyAnalysis.html
[1] https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/42958.pdf
 2022-06-20 16:13:25 +02:00
mergify[bot]
f3264e50ec
Merge branch 'master' into play-charts 2022-06-20 14:01:47 +00:00
mergify[bot]
2d25daf64d
Merge branch 'master' into clickhouse-disks-fixes 2022-06-20 11:20:59 +00:00
Kseniia Sumarokova
a756b4be27
Merge pull request #37391 from azat/insert-profile-events-fix 
Send profile events for INSERT queries (previously only SELECT was supported)
 2022-06-20 12:16:29 +02:00
lingpeng0314
1d463718ca Change getKeeperDispatcher error type 2022-06-20 17:11:15 +08:00
Alexey Milovidov
240d896eb4 Add primitive charting capabilities 2022-06-18 18:20:53 +02:00
Alexey Milovidov
eba897d4ec Add primitive charting capabilities 2022-06-18 18:15:45 +02:00
Maksim Kita
6b748f2193 AArch64 enable trace collection 2022-06-17 17:53:51 +02:00
kssenii
7520bfeab7 Fix 2022-06-16 19:50:09 +02:00
alesapin
ca33ff93cb
Merge pull request #37555 from ClickHouse/revert-37534-revert-37036-keeper-preprocess-operations 
Add support for preprocessing ZooKeeper operations and real-time digest in `clickhouse-keeper`
 2022-06-16 17:29:00 +02:00
Azat Khuzhin
3b29db6e9f LocalServer: remove superfluous ProgressIndicator.h 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-06-16 11:59:01 +03:00
jasperzhu
bca18b2caf
Merge branch 'master' into dev_intel_iaa_deflate 2022-06-16 08:56:27 +08:00
Robert Schulze
b936abe813
+ comments about keeping stuff in sync 2022-06-16 00:07:59 +02:00
Robert Schulze
bc46cef63c
Minor follow-up 
- change ELF section name to ".clickhouse.hash" (lowercase seems
  standard)

- more expressive/concise integrity check messages at startup
 2022-06-14 08:52:13 +00:00
Antonio Andelic
e5504f1b33 Merge branch 'master' into revert-37534-revert-37036-keeper-preprocess-operations 2022-06-13 15:43:10 +00:00
Robert Schulze
bc6f30fd40
Move binary hash to ELF section ".ClickHouse.hash" 2022-06-13 08:46:23 +00:00
Robert Schulze
1a0b5f33b3
More consistent use of platform macros 
cmake/target.cmake defines macros for the supported platforms, this
commit changes predefined system macros to our own macros.

__linux__ --> OS_LINUX
__APPLE__ --> OS_DARWIN
__FreeBSD__ --> OS_FREEBSD
 2022-06-10 10:22:31 +02:00
Anton Popov
df6882d2b9
Revert "Fix errors of CheckTriviallyCopyableMove type" 2022-06-07 13:53:10 +02:00
Varinara
1f8fab41f1 Fixe2: allow config path at any position 2022-06-06 16:52:58 +03:00
Varinara
6aa9e5d649 Fixe: allow config path at any position 2022-06-06 16:52:58 +03:00
Varinara
e35d1460bf Corrections 2022-06-06 16:52:58 +03:00
Varinara
a330d3cb4a Corrections 2022-06-06 16:52:58 +03:00
Varinara
f5d4e9ccc1 Corrections 2022-06-06 16:52:58 +03:00
Varinara
354ef2e3b8 Corrections 2022-06-06 16:52:58 +03:00
Varinara
51409703e0 Corrections 2022-06-06 16:52:58 +03:00
Varinara
ed6e8176fe Add basic commands for disk tool (list-disks, list, move, remove, link, copy, read, write) + tests 2022-06-06 16:52:58 +03:00
Antonio Andelic
2c37fe3d7b Merge branch 'master' into revert-37534-revert-37036-keeper-preprocess-operations 2022-06-06 12:39:55 +00:00
Robert Schulze
2d87af2a15
Merge pull request #37647 from DevTeamBK/Fix-all-CheckTriviallyCopyableMove-Errors 
Fix errors of CheckTriviallyCopyableMove type
 2022-06-05 19:58:47 +02:00
Alexey Milovidov
1529d47207
Merge pull request #34754 from ClickHouse/llvm-14 
Switch to clang/llvm 14
 2022-06-03 14:07:34 +03:00
Nikolai Kochetov
bc166e4394
Merge pull request #37543 from ClickHouse/refactor-read-metrics-and-callbacks 
Refactor read metrics and callbacks
 2022-06-03 09:43:25 +02:00
HeenaBansal2009
e3080f2a97 Merge remote-tracking branch 'origin' into Fix-all-CheckTriviallyCopyableMove-Errors 2022-06-02 07:30:08 -07:00
Antonio Andelic
7ee47fc5bb Merge branch 'master' into revert-37534-revert-37036-keeper-preprocess-operations 2022-06-02 13:38:52 +00:00
mergify[bot]
f678fdb0c2
Merge branch 'master' into tonickkozlov/odbc-disable-connection-pooling 2022-06-02 13:08:30 +00:00
Nikolai Kochetov
edac3d6714 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-06-02 09:36:20 +00:00
Alexey Milovidov
b5f48a7d3f Merge branch 'master' of github.com:ClickHouse/ClickHouse into llvm-14 2022-06-01 22:09:58 +02:00
Alexey Milovidov
bac09770f1 Fix clang-tidy-14 2022-06-01 22:07:26 +02:00
Nikolai Kochetov
6e924cdc77 Fix some more tests. 2022-06-01 15:21:47 +00:00
Dmitry Novik
7fbe91ca81
Merge pull request #37460 from ClickHouse/memory-overcommit-improvement 
Memory Overcommit: update defaults, exception message and add ProfileEvent
 2022-06-01 17:06:33 +02:00
mergify[bot]
4aefcf29e3
Merge branch 'revert-37534-revert-37036-keeper-preprocess-operations' into keeper-real-time-digest 2022-06-01 10:17:49 +00:00
Anton Kozlov
e9d645168e Renamed ODBCConnectionFactory to ODBCPooledConntionFactory 2022-06-01 09:00:39 +00:00
Nikolai Kochetov
86fbb74703 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-31 18:07:47 +00:00
Roman Vasin
b7cde2c5ad Remove total_max_threads logic from QueryPipelineBuilder; Add setMaxConcurrency(total_max_threads) in Server.cpp 2022-05-31 19:43:15 +03:00
Anton Kozlov
3576625647 CLICKHOUSE-2131 Add an option to disable connection pooling in ODBC bridge 2022-05-31 16:26:08 +00:00
Alexey Milovidov
4bb04f913f Fix clang-tidy-14 2022-05-31 17:20:07 +02:00
mergify[bot]
ba49c6bb46
Merge branch 'master' into memory-overcommit-improvement 2022-05-31 13:17:06 +00:00
Roman Vasin
441bd3e81f Merge remote-tracking branch 'origin/concurrency-control' into ADQM-445 2022-05-31 15:07:02 +03:00
Sergei Trifonov
026e073b0b minor improvement 2022-05-31 13:50:09 +02:00
HeenaBansal2009
1823f736cc Some more CheckTriviallyCopyableMove fix 2022-05-31 04:11:11 -07:00
Sergei Trifonov
7e95bf31b2 more verbose sanity checks 2022-05-31 09:26:26 +02:00
jasperzhu
fc03af2259
Merge branch 'master' into dev_intel_iaa_deflate 2022-05-30 17:49:04 +08:00
Nikolai Kochetov
5b4658aa5e Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-30 09:47:35 +00:00
Dmitry Novik
60b9d81773 Remove global_memory_usage_overcommit_max_wait_microseconds 2022-05-27 16:30:29 +00:00
Azat Khuzhin
dc9ca3d70c
Fix LOGICAL_ERROR in getMaxSourcePartsSizeForMerge during merges (#37413) 2022-05-26 14:14:58 +02:00
Nikolai Kochetov
84f97b53de Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-26 11:07:45 +00:00
Alexey Milovidov
f321925032
Merge pull request #36341 from ClickHouse/allow-setuid-inside-clickhouse 
Allow to drop privileges at startup
 2022-05-26 01:07:04 +03:00
Nikolai Kochetov
1b85f2c1d6 Merge branch 'master' into refactor-read-metrics-and-callbacks 2022-05-25 16:27:40 +02:00
Antonio Andelic
fe72cd7478 Merge branch 'master' into keeper-real-time-digest 2022-05-25 07:41:31 +00:00
Alexey Milovidov
0b5cfa4094 Fix split build 2022-05-25 04:24:02 +02:00
Alexey Milovidov
4556904dae Fix stress test 2022-05-25 04:21:25 +02:00
guykohen
938e766d7e Remove height restrictions from the query div in play web tool, and make sure width of the query box won't shrink below 100%. 
Default height is set to 20% which should be big enough for medium queries.
 2022-05-24 20:32:13 -04:00
guykohen
0036b1a6ee Remove height restrictions from the query div in play web tool, and make sure width of the query box won't shrink below 100%. 2022-05-24 18:24:54 -04:00
Alexey Milovidov
d773bd610d
Revert "Remove height restrictions from the query div in play web tool, and m…" 2022-05-25 00:58:57 +03:00
Alexey Milovidov
516fba27dc Merge branch 'master' into allow-setuid-inside-clickhouse 2022-05-24 23:31:14 +02:00