thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 03:12:43 +00:00
Code Issues Packages Projects Releases Wiki Activity
97,706 Commits 9,326 Branches 1,783 Tags 2.1 GiB
f4a48cd4d6
Commit Graph

4441 Commits

Author SHA1 Message Date
Vitaly Baranov
31ed722572
Merge pull request #41044 from vitlibar/more-conventional-conversion-yaml-to-xmk 
More conventional conversion yaml to xml
 2022-09-07 13:46:32 +02:00
Frank Chen
fc05b05be3 Fix style and typo 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-09-07 15:14:43 +08:00
Frank Chen
de8f6bdce7 More safe 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-09-07 13:39:12 +08:00
Vitaly Baranov
63e992d52d Edit test configs. 2022-09-06 17:09:26 +02:00
Frank Chen
6ced4131ca exception safe 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-09-06 22:11:47 +08:00
Kseniia Sumarokova
3558361a05
Merge branch 'master' into refactor-merge-tree-read 2022-09-06 16:00:43 +02:00
Suzy Wang
cc88679a52
Merge branch 'master' into sensitive_data_masker_unittest_issue 2022-09-06 09:36:53 -04:00
Vitaly Baranov
6b55f4dd68 Use pretty-print to output preprocessed configs in readable form. 2022-09-06 15:01:47 +02:00
Vitaly Baranov
e9b75deeba Make conversion YAML->XML more conventional. 2022-09-06 15:01:41 +02:00
Sergei Trifonov
f77809ddbc
Merge pull request #40900 from ClickHouse/s3-detailed-metrics 
S3 detailed metrics
 2022-09-06 13:20:57 +02:00
Alexey Milovidov
940a53e519
Merge pull request #40984 from Lucky-Chang/typo_fix 
Fix some typos and clang-tidy warnings
 2022-09-06 02:37:50 +03:00
kssenii
83514fa2ef Refactor 2022-09-05 20:08:22 +02:00
Sergei Trifonov
1c247308f1
Merge branch 'master' into s3-detailed-metrics 2022-09-05 15:55:41 +02:00
Alexander Tokmakov
abffd4595e
Merge branch 'master' into zookeeper_client_fault_injection 2022-09-05 15:03:26 +03:00
Antonio Andelic
e07539d1b9 Merge branch 'master' into keeper-storage 2022-09-05 09:54:13 +00:00
Luck-Chang
0800c0fd4c fix some typos 2022-09-05 14:09:42 +08:00
Luck-Chang
1ac8e739c9 fix some typos and clang-tidy warnings 2022-09-05 09:50:24 +08:00
Alexey Milovidov
d20d49f39f
Merge pull request #40958 from arenadata/ADQM-566 
stylecheck renewed and version in docker set explicitely
 2022-09-05 03:23:38 +03:00
kssenii
48dc32faf8 Better test 2022-09-04 19:54:34 +02:00
Alexey Milovidov
dbca269ec7
Merge pull request #40321 from azat/mem/untracked_memory_limit_increase 
Remove ThreadStatus::untracked_memory_limit_increase
 2022-09-04 04:42:24 +03:00
Alexey Milovidov
660c1439ec Fix build 2022-09-04 03:27:55 +02:00
Alexey Milovidov
13a129bee7 Merge branch 'master' into remove-useless-line 2022-09-04 03:24:40 +02:00
Alexey Milovidov
22f7bfdcb5
Update gtest_sensitive_data_masker.cpp 2022-09-04 03:42:58 +03:00
Alexey Milovidov
c6f7ddc5e6
Merge branch 'master' into sensitive_data_masker_unittest_issue 2022-09-04 03:42:24 +03:00
Azat Khuzhin
9479e2143f Do not try to load empty debug files 
This will avoid CANNOT_PARSE_ELF error for builds that has empty debug
file in clickhouse-common-static-dbg package, i.e. debug build.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-02 23:23:05 +02:00
Azat Khuzhin
597197c01d Improve error messages in Elf loader 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-02 23:22:25 +02:00
Azat Khuzhin
45afaa6fb8 Fix loading external symbols 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-02 23:22:25 +02:00
Alexander Tokmakov
2044470895 review fixes 2022-09-02 19:18:44 +02:00
Suzy Wang
83b029e716 re arrange sensitive data masker unit test scenario order 2022-09-02 10:11:57 -07:00
Sergei Trifonov
067e712fdf
Merge branch 'master' into s3-detailed-metrics 2022-09-02 16:59:52 +02:00
Ilya Golshtein
74561b40ae stylecheck renewed 2022-09-02 17:42:18 +03:00
Antonio Andelic
b816d46520
Merge branch 'master' into fix-keeper-system-path-check 2022-09-02 13:42:16 +02:00
Antonio Andelic
9ca20b97c2
Merge branch 'master' into keeper-storage 2022-09-02 12:13:07 +02:00
Alexander Tokmakov
3582a51a72
Merge pull request #40920 from ClickHouse/fix-typos 
Fix typos found by new `codespell`
 2022-09-02 13:11:16 +03:00
Alexander Tokmakov
5b6a8677ef
Merge pull request #40890 from arthurpassos/patch-1 
CaresPTRResolver small safety improvement
 2022-09-02 12:29:23 +03:00
Antonio Andelic
e64436fef3 Fix typos with new codespell 2022-09-02 08:54:48 +00:00
Antonio Andelic
b7eebfc626 Correctly check if the node is using system path 2022-09-02 07:47:12 +00:00
Robert Schulze
319d8b00a7
Merge pull request #39010 from FrankChen021/tracing_context_propagation 
Improve the opentelemetry tracing context propagation across threads
 2022-09-02 07:56:43 +02:00
Sergei Trifonov
29e922f05c support all profile events for disk s3 2022-09-02 02:22:52 +02:00
Sergei Trifonov
ac9dc53c73 Merge branch 'master' into s3-detailed-metrics 2022-09-01 18:31:02 +02:00
Sergei Trifonov
bcb6475c4a add separate s3 profile events for disk s3 2022-09-01 18:30:55 +02:00
Dmitry Novik
2ca619d40f
Merge pull request #40320 from azat/memory-tracker-fix-peak-race 
Fix rare race in updating peak of memory tracker
 2022-09-01 18:07:33 +02:00
Frank Chen
9d63cbe811 Merge 'origin/master' into tracing_context_propagation to resolve conflicts 2022-09-01 23:18:59 +08:00
Alexander Tokmakov
04d431793b
Update src/Common/ZooKeeper/ZooKeeperImpl.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2022-09-01 17:12:22 +03:00
Alexander Tokmakov
36454a340d
Update src/Common/ZooKeeper/ZooKeeperArgs.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2022-09-01 17:12:16 +03:00
Antonio Andelic
62ff61404f Improve remove recursive 2022-09-01 12:16:13 +00:00
Arthur Passos
fb42afbbac
CaresPTRResolver small safety improvement 
Previous to #40769, only `hostent::h_aliases` was being accessed. After that PR got merged, `hostent::h_name` started being accessed as well. This PR moves the first `hostent::h_aliases != nullptr` check that could prevent `hostent::h_name` from being accessed. During debugging, I observed that even when there are not aliases, `hostent::h_aliases` is not null. That's why it hasn't caused any problems, but proposing this change to be on the safe side.
 2022-09-01 08:59:14 -03:00
kssenii
82879543d3 Merge remote-tracking branch 'upstream/master' into improve-marks-cache-loading 2022-09-01 13:13:01 +02:00
Antonio Andelic
c7d1dd942c Address PR comments 2022-09-01 07:56:06 +00:00
Frank Chen
d9a516288c Resolve comments 2022-09-01 11:56:10 +08:00
kssenii
10a7e73d52 Fix unit test 2022-08-31 21:07:39 +02:00
kssenii
0f747e7987 Move cache files 2022-08-31 17:50:55 +02:00
Robert Schulze
ca01286028
Merge pull request #39629 from ClickHouse/catboost-bridge 
Move CatBoost evaluation into clickhouse-library-bridge
 2022-08-31 16:16:11 +02:00
Antonio Andelic
9dd1a9859d Merge branch 'master' into keeper-storage 2022-08-31 10:12:11 +00:00
Kseniia Sumarokova
9abcfce529
Merge pull request #40759 from kssenii/fix-write-through-cache-logical-error 
Fix logical error from write-though cache from stress test
 2022-08-31 11:55:18 +02:00
kssenii
a5f69517ae Allow to load marks in threadpool 2022-08-30 20:20:05 +02:00
Alexander Tokmakov
6a50c20734
Merge branch 'master' into zookeeper_client_fault_injection 2022-08-30 15:43:30 +03:00
Alexander Tokmakov
022f440ad0
Merge pull request #40769 from arthurpassos/caresptrresolver-hosts-patch 
Fix CaresPTRResolver not reading hosts file
 2022-08-30 14:35:10 +03:00
Kseniia Sumarokova
54f6f5bb2c
Merge branch 'master' into fix-write-through-cache-logical-error 2022-08-30 13:30:13 +02:00
Kseniia Sumarokova
5eb188311a
Merge pull request #40586 from kssenii/fix-race 
Fix possible race in fs cache
 2022-08-30 12:58:31 +02:00
Alexander Tokmakov
0ec7f068cc
Merge pull request #40677 from ClickHouse/query-insert-oom 
Avoid deadlock in case of new query and OOM
 2022-08-30 13:54:34 +03:00
Antonio Andelic
53da469147 Merge branch 'master' into keeper-storage 2022-08-30 10:22:04 +00:00
Antonio Andelic
80365e0448 Simplify drop and create 2022-08-30 09:19:59 +00:00
Frank Chen
922a0dee74 Fix style 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-08-30 15:14:12 +08:00
Frank Chen
f17d56b528 Merge branch 'master' into tracing_context_propagation 2022-08-30 14:24:36 +08:00
Frank Chen
83cbdef3c6 Fix context re-initialization for ThreadPool 2022-08-30 12:26:23 +08:00
Robert Schulze
6b2b3c1eb3
feat: implement catboost in library-bridge 
This commit moves the catboost model evaluation out of the server
process into the library-bridge binary. This serves two goals: On the
one hand, crashes / memory corruptions of the catboost library no longer
affect the server. On the other hand, we can forbid loading dynamic
libraries in the server (catboost was the last consumer of this
functionality), thus improving security.

SQL syntax:

  SELECT
    catboostEvaluate('/path/to/model.bin', FEAT_1, ..., FEAT_N) > 0 AS prediction,
    ACTION AS target
  FROM amazon_train
  LIMIT 10

Required configuration:

  <catboost_lib_path>/path/to/libcatboostmodel.so</catboost_lib_path>

*** Implementation Details ***

The internal protocol between the server and the library-bridge is
simple:

- HTTP GET on path "/extdict_ping":
  A ping, used during the handshake to check if the library-bridge runs.

- HTTP POST on path "extdict_request"
  (1) Send a "catboost_GetTreeCount" request from the server to the
      bridge, containing a library path (e.g /home/user/libcatboost.so) and
      a model path (e.g. /home/user/model.bin). Rirst, this unloads the
      catboost library handler associated to the model path (if it was
      loaded), then loads the catboost library handler associated to the
      model path, then executes GetTreeCount() on the library handler and
      finally sends the result back to the server. Step (1) is called once
      by the server from FunctionCatBoostEvaluate::getReturnTypeImpl(). The
      library path handler is unloaded in the beginning because it contains
      state which may no longer be valid if the user runs
      catboost("/path/to/model.bin", ...) more than once and if "model.bin"
      was updated in between.
  (2) Send "catboost_Evaluate" from the server to the bridge, containing
      the model path and the features to run the interference on. Step (2)
      is called multiple times (once per chunk) by the server from function
      FunctionCatBoostEvaluate::executeImpl(). The library handler for the
      given model path is expected to be already loaded by Step (1).

Fixes #27870
 2022-08-29 20:26:45 +00:00
Dmitry Novik
865ee5d0d6 Refactor code 2022-08-29 20:24:35 +02:00
Arthur Passos
961365c7a4 Fix CaresPTRResolver not reading hosts file 2022-08-29 15:11:39 -03:00
Kseniia Sumarokova
c5c48e44ea
Merge branch 'master' into fix-mysql-timeouts 2022-08-29 19:33:29 +02:00
kssenii
545c6c8be4 Fix 2022-08-29 17:50:27 +02:00
kssenii
b1dab84d97 Review fixes 2022-08-29 16:23:14 +02:00
kssenii
0a6c4b9265 Fix 2022-08-29 16:20:53 +02:00
kssenii
877ade9a50 Merge remote-tracking branch 'upstream/master' into fix-race 2022-08-29 16:05:27 +02:00
Alexander Tokmakov
8bdb589c2b Merge branch 'master' into zookeeper_client_fault_injection 2022-08-29 13:34:57 +02:00
Alexander Tokmakov
69387acffa
Revert "Support for DWARF-5 in in house DWARF parser" 2022-08-29 14:25:53 +03:00
Alexey Milovidov
ae97e880d7
Merge pull request #40710 from azat/DWARF-5 
Support for DWARF-5 in in house DWARF parser
 2022-08-29 00:13:33 +03:00
Alexey Milovidov
f462160bd1 Merge remote-tracking branch 'origin/master' into remove-useless-line 2022-08-28 21:00:29 +02:00
Alexey Milovidov
00074a5e14
Merge pull request #40711 from ClickHouse/all_new_functions_must_be_documented 
All new functions must be documented
 2022-08-28 17:05:05 +03:00
Azat Khuzhin
e03870bc8b Common/Dwarf: fix clang-tidy 2022-08-28 15:51:46 +02:00
Alexey Milovidov
d3a7945b1b
Merge pull request #40646 from ClickHouse/force-documentation-3 
Every profile event must have documentation
 2022-08-28 04:10:11 +03:00
Alexey Milovidov
b42d26acfe Remove one line from XML, because we do not care 2022-08-28 02:44:02 +02:00
Azat Khuzhin
ee5696bb32 Add support for DWARF-5 
I have to do this, since I have libraries compiled with DWARF5 (i.e.
glibc).

ClickHouse changes:
- use camel_case
- add NOLINT
- avoid using folly:: (use std:: instead)
- avoid using boost:: (use std:: instead)

Refs: 490b287ca3
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-27 22:28:15 +02:00
Alexey Milovidov
6139cc8f7a Add Documentation to FunctionFactory 2022-08-27 22:06:57 +02:00
Alexey Milovidov
e7054debc1
Merge pull request #40649 from ClickHouse/embedded-documentation 
A prototype for embedded documentation
 2022-08-27 21:28:11 +03:00
Alexey Milovidov
a6387ac0f7
Merge branch 'master' into force-documentation-3 2022-08-27 19:31:44 +03:00
Alexey Milovidov
e04f69717b
Merge pull request #40673 from ClickHouse/vdimir/tmp-file-metrics-doc 
Add doc to ExternalSort/Agg/Join ProfileEvents
 2022-08-27 16:52:40 +03:00
Alexander Gololobov
89be7a5a25
Update ProfileEvents.cpp 2022-08-26 22:55:42 +02:00
Kseniia Sumarokova
11b82ffc5e
Merge branch 'master' into fix-race 2022-08-26 22:55:14 +02:00
Kseniia Sumarokova
0cf76dab77
Merge pull request #40608 from ClickHouse/kssenii-patch-4 
Fix stress test after #40420
 2022-08-26 22:54:37 +02:00
Alexey Milovidov
c9858099ea
Update ProfileEvents.cpp 2022-08-26 22:10:10 +03:00
Alexander Gololobov
4fef408104
Merge branch 'master' into force-documentation-3 2022-08-26 20:29:06 +02:00
vdimir
c5a9b64080
Add doc to ExternalSort/Agg/Join ProfileEvents 2022-08-26 13:41:53 +00:00
Alexey Milovidov
46594e47bc Add comment 2022-08-26 15:28:18 +02:00
kssenii
470e18677f Ger rid of cache lock 2022-08-26 14:13:36 +02:00
Vladimir C
e067629e0d
Merge pull request #40239 from vdimir/vdimir/tmp-file-metrics 
More metrics for on-disk temporary data
 2022-08-26 11:28:01 +02:00
Robert Schulze
1c8c83ccf3
Merge pull request #40620 from zvonand/zvonand-b58 
Base58 fix handling leading 0 / '1'
 2022-08-26 10:18:26 +02:00
Alexander Gololobov
c7a792d929
small grammar fix 2022-08-26 10:03:43 +02:00
Alexander Gololobov
856f83caa1
Removed unnecessary commas 2022-08-26 10:00:20 +02:00
Frank Chen
cf1081eada Exception safe from destructor 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-08-26 11:18:16 +08:00