Commit Graph

97656 Commits

Author SHA1 Message Date
Denny Crane
132727d212
Doc. Fix links. 2022-09-08 14:37:42 -03:00
Dmitry Novik
29b5d023ee Replace WITH_TEXT_LOG by WITHOUT_TEXT_LOG 2022-09-08 19:16:37 +02:00
zhenjial
469ceaa156 code optimization 2022-09-09 00:47:43 +08:00
Vitaly Baranov
e48cd2b6f6 Add more test cases. 2022-09-08 18:14:25 +02:00
Vitaly Baranov
9acc73d811
Update src/Storages/StorageReplicatedMergeTree.cpp
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
2022-09-08 17:51:29 +02:00
alesapin
62c911924b
Merge pull request #41108 from ClickHouse/more_logging
More logging for S3
2022-09-08 17:10:55 +02:00
alesapin
5e00124f8a More logging for S3 2022-09-08 16:18:21 +02:00
Alexey Milovidov
928c1cd0d4
Merge pull request #40967 from ClickHouse/alexey-milovidov-patch-4
Update tips.md about filesystems
2022-09-08 16:56:27 +03:00
Antonio Andelic
3502718a2c Less complications 2022-09-08 13:52:14 +00:00
Vincent Bernat
fab1b40928 docs: mention SYNC modifier for DROP and DETACH statements 2022-09-08 15:21:59 +02:00
Antonio Andelic
b6f8e06708
Typo fix 2022-09-08 14:57:34 +02:00
Alexander Tokmakov
507ad0c4d9
Merge pull request #40691 from azat/mergetree/transaction-wal-fix
Move committing InMemory parts to WAL out of NOEXCEPT_SCOPE()
2022-09-08 15:55:30 +03:00
Antonio Andelic
b99996961d Address PR comments 2022-09-08 12:45:36 +00:00
vdimir
8121904dc4
Minor update StorageEmbeddedRocksDB::getSampleBlock 2022-09-08 12:43:38 +00:00
alesapin
15dc68340f
Merge pull request #41103 from ClickHouse/return_key_to_message
Improve logging one time
2022-09-08 14:00:06 +02:00
Vitaly Baranov
122009a2bd Use table lock if database is ordinary and zero-copy-replication is enabled. 2022-09-08 13:54:59 +02:00
Kruglov Pavel
04aea19b31
Merge pull request #40881 from coelho/patch-1
Fix uncaught DNS_ERROR on failed connection to replicas
2022-09-08 13:32:45 +02:00
Maksim Kita
1f9ea15aab
Merge pull request #41049 from canhld94/ch_canh_gs_core
Remove parser logic for WITH GROUPING SETS
2022-09-08 13:32:27 +02:00
Kruglov Pavel
96cb0e54d8
Merge branch 'master' into Avogar-patch-1 2022-09-08 13:24:08 +02:00
alesapin
6da37b92e1 Improve logging one time 2022-09-08 13:11:53 +02:00
Kseniia Sumarokova
7f086a94a7
Update storing-data.md 2022-09-08 12:34:58 +02:00
Vladimir C
90647cb0c5
Merge pull request #39970 from SachinSetiya/majority_quorum
Add majority_insert_quorum setting
2022-09-08 11:31:33 +02:00
Frank Chen
a986380522
Update src/Storages/Distributed/DirectoryMonitor.cpp
Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com>
2022-09-08 17:25:29 +08:00
Robert Schulze
c16707ff00
chore: delete obsolete modelEvaluate() function + SYSTEM.MODELS view
- The deleted function modelEvaluate() was superseded by
  catboostEvaluate().

- Also delete the external model repository, as modelEvaluate() was it's
  last user. Additionally remove the system view SYSTEM.MODELS for
  inspecting the repository.

- SYSTEM RELOAD MODELS is also obsolete. HOWEVER, it was retained and
  made a no-op instead of deleted.

  Why?
  The reason is that RBAC in distributed setups works by storing
  privileges (granted and revoked) as plain SQL statements in Keeper.
  Nodes read these statements at startup and parse them. If a privilege
  for SYSTEM RELOAD MODELS exists but parser doesn't recognize it
  nodes would fail to come up.

  Considered but rejected alternatives:
  - Ignore SYSTEM RELOAD MODELS during parsing RBAC privileges and
    return an error for regular SYSTEM RELOAD MODELS SQL. Special-case
    of no-op behavior, too brittle.
  - Remove SYSTEM RELOAD MODELS manually from Keeper via command-line
    manipulation of Keeper nodes or via SQL by dropping the privileges.
    Needs user intervention during upgrade.
2022-09-08 09:10:11 +00:00
Robert Schulze
60f9f6855d
feat: implement catboost in library-bridge
This commit moves the catboost model evaluation out of the server
process into the library-bridge binary. This serves two goals: On the
one hand, crashes / memory corruptions of the catboost library no longer
affect the server. On the other hand, we can forbid loading dynamic
libraries in the server (catboost was the last consumer of this
functionality), thus improving security.

SQL syntax:

  SELECT
    catboostEvaluate('/path/to/model.bin', FEAT_1, ..., FEAT_N) > 0 AS prediction,
    ACTION AS target
  FROM amazon_train
  LIMIT 10

Required configuration:

  <catboost_lib_path>/path/to/libcatboostmodel.so</catboost_lib_path>

*** Implementation Details ***

The internal protocol between the server and the library-bridge is
simple:

- HTTP GET on path "/extdict_ping":
  A ping, used during the handshake to check if the library-bridge runs.

- HTTP POST on path "extdict_request"
  (1) Send a "catboost_GetTreeCount" request from the server to the
      bridge, containing a library path (e.g /home/user/libcatboost.so) and
      a model path (e.g. /home/user/model.bin). Rirst, this unloads the
      catboost library handler associated to the model path (if it was
      loaded), then loads the catboost library handler associated to the
      model path, then executes GetTreeCount() on the library handler and
      finally sends the result back to the server. Step (1) is called once
      by the server from FunctionCatBoostEvaluate::getReturnTypeImpl(). The
      library path handler is unloaded in the beginning because it contains
      state which may no longer be valid if the user runs
      catboost("/path/to/model.bin", ...) more than once and if "model.bin"
      was updated in between.
  (2) Send "catboost_Evaluate" from the server to the bridge, containing
      the model path and the features to run the interference on. Step (2)
      is called multiple times (once per chunk) by the server from function
      FunctionCatBoostEvaluate::executeImpl(). The library handler for the
      given model path is expected to be already loaded by Step (1).

Fixes #27870
2022-09-08 09:01:32 +00:00
Robert Schulze
0df6173e4a
Merge remote-tracking branch 'origin/master' into regen-vectorscan-assets 2022-09-08 08:37:30 +00:00
Mikhail f. Shiryaev
bd10a2195b
Build macos binaries in backport CI, add BuilderSpecialReport to Finish 2022-09-08 10:33:24 +02:00
Antonio Andelic
09c0bf2931 Add unit tests for match path 2022-09-08 08:16:38 +00:00
Antonio Andelic
5d837e4f41 Merge branch 'master' into fix-keeper-system-path-check 2022-09-08 07:41:04 +00:00
Robert Schulze
68808858a5
Merge pull request #41050 from FrankChen021/exception_safe
Fix failed stress test (OpenTelemetry)
2022-09-08 09:19:54 +02:00
Robert Schulze
9d4de0cbaa
Merge pull request #40999 from ClickHouse/sse2-special-build
Add special x86-SSE2-only build
2022-09-08 09:06:29 +02:00
Frank Chen
329f31e7ab Address review comments
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-09-08 11:38:10 +08:00
Anton Popov
7c12b448b8 Merge remote-tracking branch 'upstream/master' into HEAD 2022-09-08 01:52:52 +00:00
Anton Popov
5a790b15b4 try to fix filling of missed Nested columns with multiple levels 2022-09-08 00:20:11 +00:00
Alexey Milovidov
9544b8fdd6
Merge pull request #40996 from ClickHouse/vdimir/issue-40994
Minor update doc for mysql_port
2022-09-08 02:39:12 +03:00
Alexey Milovidov
84a00e3992
Merge pull request #41087 from peter279k/improve_clickhouse_start
Improve clickhouse start command
2022-09-08 02:35:02 +03:00
Suzy Wang
61b2e1c32c
Merge branch 'master' into sensitive_data_masker_unittest_issue 2022-09-07 16:49:03 -04:00
Kseniia Sumarokova
eb53df48d1
Update storing-data.md 2022-09-07 22:26:52 +02:00
Kseniia Sumarokova
3af51f4340
Update storing-data.md 2022-09-07 22:21:46 +02:00
Robert Schulze
9b868f8b7d
doc: add note about upstream colm/ragel 2022-09-07 20:10:17 +00:00
Robert Schulze
151bf1e074
fix: suppress warnings for ARM ragel assets 2022-09-07 20:08:31 +00:00
Nikolay Degterinsky
5f6699ab1e
Merge pull request #41093 from den-crane/patch-46
Doc. update date_diff
2022-09-07 21:23:02 +02:00
Igor Nikonov
e74bb00e90 Fix: EXPLAIN PLAN - make settings in SETTING clause effective 2022-09-07 19:11:25 +00:00
Denny Crane
a75eb5ad84
Update date-time-functions.md 2022-09-07 15:59:23 -03:00
Yuko Takagi
fb6b26c7a4
Update README.md (#41091) 2022-09-07 20:58:36 +02:00
Denny Crane
0071ef9e38
Update date-time-functions.md 2022-09-07 15:56:31 -03:00
peter279k
945299de99 Remove strange release trains 2022-09-08 01:24:12 +08:00
Mikhail f. Shiryaev
c0ce4c2d6f
Add macos binaries to GH release assets 2022-09-07 19:23:44 +02:00
Mikhail f. Shiryaev
a067907fbe
Add typing and order import 2022-09-07 19:23:43 +02:00
Mikhail f. Shiryaev
bd83e905d3
Fix a typo in download_build_with_progress 2022-09-07 19:23:43 +02:00