Commit Graph

96792 Commits

Author SHA1 Message Date
Robert Schulze
c185353bf0
Minor: remove unnecessary ContextAccess::make() method 2022-08-30 08:06:42 +00:00
Robert Schulze
cc4225109f
Merge pull request #37215 from Vector-Similarity-Search-for-ClickHouse/annoy-2
Test failures are unrelated, merging.
2022-08-30 09:25:57 +02:00
Robert Schulze
566daad4a1
Merge pull request #40723 from k-morozov/ref/remove-unnecessary-allocation
refactoring: remove unnecessary allocation
2022-08-30 09:22:21 +02:00
Robert Schulze
8ba9f6dd5d
fix: delete obsolete test 02413_model_evaluate_smoke 2022-08-30 07:21:05 +00:00
Frank Chen
922a0dee74 Fix style
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-08-30 15:14:12 +08:00
Frank Chen
f17d56b528 Merge branch 'master' into tracing_context_propagation 2022-08-30 14:24:36 +08:00
Frank Chen
83cbdef3c6 Fix context re-initialization for ThreadPool 2022-08-30 12:26:23 +08:00
Yakov Olkhovskiy
a9cab86a73 typo 2022-08-30 01:46:17 +00:00
Yakov Olkhovskiy
7a85a943f1 fix close lock on exec, refactoring, comments added 2022-08-30 01:11:04 +00:00
Cory Levy
cd371110ad Fix timezone offset formatting 2022-08-29 21:01:27 -04:00
Sema Checherinda
e436b4f4cc Metric for the number of detached parts 2022-08-30 02:12:57 +02:00
DanRoscigno
d712a91a20 add alternatives 2022-08-29 19:36:20 -04:00
DanRoscigno
0abeebd3ca updated with dev help 2022-08-29 19:29:10 -04:00
Alexey Milovidov
d5fae3d16b
Merge pull request #40752 from ClickHouse/auto/v22.8.3.13-lts
Update version_date.tsv and changelogs after v22.8.3.13-lts
2022-08-30 02:21:55 +03:00
Alexey Milovidov
c4e664a899
Merge pull request #40753 from ClickHouse/auto/v22.7.5.13-stable
Update version_date.tsv and changelogs after v22.7.5.13-stable
2022-08-30 02:21:44 +03:00
Alexey Milovidov
0af90273f9
Merge pull request #40757 from ClickHouse/auto/v22.3.12.19-lts
Update version_date.tsv and changelogs after v22.3.12.19-lts
2022-08-30 02:21:34 +03:00
Alexey Milovidov
bddbbafea0
Merge pull request #40755 from ClickHouse/auto/v22.6.7.7-stable
Update version_date.tsv and changelogs after v22.6.7.7-stable
2022-08-30 02:21:12 +03:00
Alexey Milovidov
6bc8983756
Merge pull request #40771 from den-crane/patch-41
Doc. fix description min_bytes_to_rebalance_partition_over_jbod
2022-08-30 02:20:07 +03:00
Alexey Milovidov
0190c56faf
Merge pull request #40770 from den-crane/patch-40
Doc. Fix cache dictionaries doc.
2022-08-30 02:19:32 +03:00
DanRoscigno
3a65d58c13 updated with dev help 2022-08-29 18:33:26 -04:00
alesapin
ac41de73df Add cache to zero copy replication test 2022-08-30 00:17:01 +02:00
Kseniia Sumarokova
8001560082
Fix test 2022-08-30 00:14:00 +02:00
Dan Roscigno
8bd72a7463
Merge branch 'master' into add-backup 2022-08-29 17:18:53 -04:00
Cory Levy
fe24bd9a7f Add tests 2022-08-29 16:47:33 -04:00
Cory Levy
1e2eee7146 Remove unnecessary backslashes in markdown sql blocks 2022-08-29 16:31:31 -04:00
Robert Schulze
4ae7586a2d
fix: test 02415_all_new_functions_must_be_documented
I am happy to move the documentation into the source code but currently
we'd need to maintain it twice (source code + traditional markdown docs)
because we don't render the docs from the source code.
2022-08-29 20:27:06 +00:00
Robert Schulze
4ed1e1a5be
perf: don't copy data around unnecessarily 2022-08-29 20:27:06 +00:00
Robert Schulze
35a37c91f8
chore: incorporate review feedback 2022-08-29 20:27:06 +00:00
Robert Schulze
3aec76d281
fix: test 02117_show_create_table_system 2022-08-29 20:27:06 +00:00
Robert Schulze
64a6aa328e
fix: broken links in documentation (hopefully) 2022-08-29 20:27:06 +00:00
robot-clickhouse
64fa077148
style: fix style 2022-08-29 20:27:06 +00:00
Robert Schulze
4d511332c4
chore: delete obsolete modelEvaluate() function
- superseded by catboostEvaluate() which no longer uses the internal
  repository for external models

- also removed was statement SYSTEM RELOAD MODELS and the monitoring view
  SYSTEM.SYSTEMMODELS
2022-08-29 20:27:06 +00:00
Robert Schulze
6b2b3c1eb3
feat: implement catboost in library-bridge
This commit moves the catboost model evaluation out of the server
process into the library-bridge binary. This serves two goals: On the
one hand, crashes / memory corruptions of the catboost library no longer
affect the server. On the other hand, we can forbid loading dynamic
libraries in the server (catboost was the last consumer of this
functionality), thus improving security.

SQL syntax:

  SELECT
    catboostEvaluate('/path/to/model.bin', FEAT_1, ..., FEAT_N) > 0 AS prediction,
    ACTION AS target
  FROM amazon_train
  LIMIT 10

Required configuration:

  <catboost_lib_path>/path/to/libcatboostmodel.so</catboost_lib_path>

*** Implementation Details ***

The internal protocol between the server and the library-bridge is
simple:

- HTTP GET on path "/extdict_ping":
  A ping, used during the handshake to check if the library-bridge runs.

- HTTP POST on path "extdict_request"
  (1) Send a "catboost_GetTreeCount" request from the server to the
      bridge, containing a library path (e.g /home/user/libcatboost.so) and
      a model path (e.g. /home/user/model.bin). Rirst, this unloads the
      catboost library handler associated to the model path (if it was
      loaded), then loads the catboost library handler associated to the
      model path, then executes GetTreeCount() on the library handler and
      finally sends the result back to the server. Step (1) is called once
      by the server from FunctionCatBoostEvaluate::getReturnTypeImpl(). The
      library path handler is unloaded in the beginning because it contains
      state which may no longer be valid if the user runs
      catboost("/path/to/model.bin", ...) more than once and if "model.bin"
      was updated in between.
  (2) Send "catboost_Evaluate" from the server to the bridge, containing
      the model path and the features to run the interference on. Step (2)
      is called multiple times (once per chunk) by the server from function
      FunctionCatBoostEvaluate::executeImpl(). The library handler for the
      given model path is expected to be already loaded by Step (1).

Fixes #27870
2022-08-29 20:26:45 +00:00
Vitaly Baranov
33f72fb011
Merge pull request #40060 from ClickHouse/vitlibar-increase-timeout-for-test_concurrent_backups
Increase timeout for test_concurrent_backups
2022-08-29 22:25:56 +02:00
Dan Roscigno
76a45aa750
Merge branch 'master' into add-backup 2022-08-29 16:23:53 -04:00
Dan Roscigno
8e5e1c5e8c
Merge pull request #40774 from DanRoscigno/replace-zh-symlinks
Replace zh symlinks
2022-08-29 15:36:08 -04:00
DanRoscigno
d37029dd82 updates for filename changes 2022-08-29 15:20:28 -04:00
Arthur Passos
dd49b44abb Fix host_regexp hosts file tst 2022-08-29 15:58:18 -03:00
DanRoscigno
576b7ea604 updates for filename changes 2022-08-29 14:39:15 -04:00
Dmitry Novik
e25ed9547e
Update src/Interpreters/ProcessList.h 2022-08-29 20:26:37 +02:00
Denny Crane
29e7414697
Update merge-tree-settings.md 2022-08-29 15:25:46 -03:00
Dmitry Novik
865ee5d0d6 Refactor code 2022-08-29 20:24:35 +02:00
Denny Crane
19c3a9c6bf
Update external-dicts-dict-layout.md 2022-08-29 15:20:46 -03:00
Denny Crane
fe0f18f21d
Update external-dicts-dict-layout.md 2022-08-29 15:19:15 -03:00
Arthur Passos
961365c7a4 Fix CaresPTRResolver not reading hosts file 2022-08-29 15:11:39 -03:00
DanRoscigno
687ac1805a updates for filename changes 2022-08-29 13:59:51 -04:00
Dmitry Novik
1169315580 Add OvercommitTracker blocking 2022-08-29 19:44:05 +02:00
Maksim Kita
88141cae98
Merge pull request #40732 from azat/thread-status-fix-leak
Fix memory leak while pushing to MVs w/o query context (from Kafka/...)
2022-08-29 19:36:25 +02:00
Kseniia Sumarokova
0fd961acd4
Merge branch 'master' into fix-race 2022-08-29 19:33:38 +02:00
Kseniia Sumarokova
c5c48e44ea
Merge branch 'master' into fix-mysql-timeouts 2022-08-29 19:33:29 +02:00