Commit Graph

12085 Commits

Author SHA1 Message Date
DanRoscigno
149970447a add link to formats 2022-09-16 17:21:03 -04:00
vvbufetov
fb94e43a60
Fix the link on the original article about MaterializedView (#41339)
Fix the link on the original article about MaterializedView

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2022-09-16 17:52:32 +02:00
Robert Schulze
b32b02d844
Merge pull request #40897 from ClickHouse/catboost-bridge-resurrected
Move CatBoost evaluation into clickhouse-library-bridge
2022-09-16 13:12:09 +02:00
Kseniia Sumarokova
08f00a7593
Merge pull request #41354 from ClickHouse/kssenii-patch-5
Update storing-data.md
2022-09-15 14:07:43 +02:00
Kseniia Sumarokova
c511c3f168
Merge pull request #40889 from AVMusorin/docs_table_function_update
Doc. Updated optional parameters for table functions
2022-09-15 13:12:20 +02:00
Kseniia Sumarokova
1ec7ce265b
Update storing-data.md 2022-09-15 12:30:02 +02:00
Kseniia Sumarokova
4935a4bf96
Update storing-data.md 2022-09-15 12:29:22 +02:00
Denny Crane
69d90de9a3
Update date-time-functions.md 2022-09-14 15:23:59 -03:00
Denny Crane
cbdbe1077a
Update date-time-functions.md 2022-09-14 12:16:32 -03:00
Dan Roscigno
c8dcd34abe
Merge pull request #41237 from DanRoscigno/move-title
move title to frontmatter in troubleshooting doc
2022-09-14 10:15:18 -04:00
Robert Schulze
cb04c724b9
Merge pull request #41285 from ClickHouse/fix-broken-doc-links
System statements docs: Fix broken links
2022-09-14 13:03:01 +02:00
Kseniia Sumarokova
bb5f9e578a
Update storing-data.md 2022-09-14 12:17:24 +02:00
Robert Schulze
d071dbe5a0
Merge branch 'master' into patch-1 2022-09-13 22:27:48 +02:00
Robert Schulze
49136d26aa
docs: fix internal links
The links at the very beginning of
https://clickhouse.com/docs/en/sql-reference/statements/system
don't work. They are reference other sections of the same document. This
is weird because there is a small index already on the right side. I
searched our documentation and this seems to be the only pages which do
so. Therefore removing the links altogether instead of fixing them.
2022-09-13 19:45:46 +00:00
Dan Roscigno
078b9bff3c
Update docs/en/sql-reference/statements/delete.md 2022-09-13 13:18:01 -04:00
Rich Raposa
b833c39dcf
Update delete.md 2022-09-13 10:55:26 -06:00
Rich Raposa
e85b222136
Update delete.md 2022-09-13 10:54:47 -06:00
DanRoscigno
64bd1ea77a spelling 2022-09-13 12:42:24 -04:00
DanRoscigno
fc2e7159f0 add note about privs 2022-09-13 12:40:16 -04:00
Robert Schulze
3e6821e8d7
fix: repair broken documentation links and update docs 2022-09-13 16:00:53 +00:00
robot-clickhouse
b9cb2ad7f7 Update version_date.tsv and changelogs after v22.8.5.29-lts 2022-09-13 12:31:55 +00:00
Robert Schulze
fac1be9700
chore: restore SYSTEM RELOAD MODEL(S) and moniting view SYSTEM.MODELS
- This commit restores statements "SYSTEM RELOAD MODEL(S)" which provide
  a mechanism to update a model explicitly. It also saves potentially
  unnecessary reloads of a model from disk after it's initial load.

  To keep the complexity low, the semantics of "SYSTEM RELOAD MODEL(S)
  was changed from eager to lazy. This means that both statements
  previously immedately reloaded the specified/all models, whereas now
  the statements only trigger an unload and the first call to
  catboostEvaluate() does the actual load.

- Monitoring view SYSTEM.MODELS is also restored but with some obsolete
  fields removed. The view was not documented in the past and for now it
  remains undocumented. The commit is thus not considered a breach of
  ClickHouse's public interface.
2022-09-12 19:33:02 +00:00
DanRoscigno
a3826c4b3f move title to frontmatter 2022-09-12 13:27:52 -04:00
Alexey Milovidov
f3a16ec08f Fix Chinese changelog 2022-09-10 05:17:55 +02:00
Rich Raposa
b9e9d776f0
Update delete.md 2022-09-09 15:55:57 -06:00
Rich Raposa
83df07d251
Merge branch 'master' into lightweight-delete 2022-09-09 13:59:53 -06:00
rfraposa
4af246a2e0 Feedback 2022-09-09 13:59:21 -06:00
Alexey Milovidov
10fd396738
Merge pull request #41160 from DanRoscigno/fix-duplicate-route
fix documentation slug in Chinese folder
2022-09-09 22:49:22 +03:00
DanRoscigno
9d717d62e1 fix slug 2022-09-09 14:56:25 -04:00
Aleksandr
503398ac96
Merge branch 'master' into docs_table_function_update 2022-09-09 20:54:28 +02:00
Rich Raposa
9870957621
Update docs/en/sql-reference/statements/delete.md
Co-authored-by: Ivan Blinkov <github@blinkov.ru>
2022-09-09 12:16:54 -06:00
Rich Raposa
50789126a8
Update docs/en/sql-reference/statements/delete.md
Co-authored-by: Ivan Blinkov <github@blinkov.ru>
2022-09-09 12:15:59 -06:00
Rich Raposa
34c2c4bb52
Merge branch 'master' into lightweight-delete 2022-09-09 12:13:51 -06:00
Nikolay Degterinsky
6dcaa9f14d
Merge pull request #41106 from vincentbernat/fix/docs-sync
docs: mention SYNC modifier for DROP and DETACH statements
2022-09-09 20:12:47 +02:00
DanRoscigno
afe3717761 move title to frontmatter to allow inclusion in other docs 2022-09-09 08:34:42 -04:00
rfraposa
18f5b5e5b0 Add docs for lightweight deletes 2022-09-08 16:05:56 -06:00
Alexey Milovidov
4d7097f47a
Merge pull request #41080 from kssenii/add-documentation-for-cache
Add documentation for filesystem cache
2022-09-09 00:03:16 +03:00
Alexey Milovidov
9199656175
Merge pull request #41082 from peter279k/remove_strange_train
Remove strange release trains
2022-09-09 00:01:35 +03:00
Denny Crane
132727d212
Doc. Fix links. 2022-09-08 14:37:42 -03:00
Alexey Milovidov
928c1cd0d4
Merge pull request #40967 from ClickHouse/alexey-milovidov-patch-4
Update tips.md about filesystems
2022-09-08 16:56:27 +03:00
Vincent Bernat
fab1b40928 docs: mention SYNC modifier for DROP and DETACH statements 2022-09-08 15:21:59 +02:00
Kseniia Sumarokova
7f086a94a7
Update storing-data.md 2022-09-08 12:34:58 +02:00
Vladimir C
90647cb0c5
Merge pull request #39970 from SachinSetiya/majority_quorum
Add majority_insert_quorum setting
2022-09-08 11:31:33 +02:00
Robert Schulze
c16707ff00
chore: delete obsolete modelEvaluate() function + SYSTEM.MODELS view
- The deleted function modelEvaluate() was superseded by
  catboostEvaluate().

- Also delete the external model repository, as modelEvaluate() was it's
  last user. Additionally remove the system view SYSTEM.MODELS for
  inspecting the repository.

- SYSTEM RELOAD MODELS is also obsolete. HOWEVER, it was retained and
  made a no-op instead of deleted.

  Why?
  The reason is that RBAC in distributed setups works by storing
  privileges (granted and revoked) as plain SQL statements in Keeper.
  Nodes read these statements at startup and parse them. If a privilege
  for SYSTEM RELOAD MODELS exists but parser doesn't recognize it
  nodes would fail to come up.

  Considered but rejected alternatives:
  - Ignore SYSTEM RELOAD MODELS during parsing RBAC privileges and
    return an error for regular SYSTEM RELOAD MODELS SQL. Special-case
    of no-op behavior, too brittle.
  - Remove SYSTEM RELOAD MODELS manually from Keeper via command-line
    manipulation of Keeper nodes or via SQL by dropping the privileges.
    Needs user intervention during upgrade.
2022-09-08 09:10:11 +00:00
Robert Schulze
60f9f6855d
feat: implement catboost in library-bridge
This commit moves the catboost model evaluation out of the server
process into the library-bridge binary. This serves two goals: On the
one hand, crashes / memory corruptions of the catboost library no longer
affect the server. On the other hand, we can forbid loading dynamic
libraries in the server (catboost was the last consumer of this
functionality), thus improving security.

SQL syntax:

  SELECT
    catboostEvaluate('/path/to/model.bin', FEAT_1, ..., FEAT_N) > 0 AS prediction,
    ACTION AS target
  FROM amazon_train
  LIMIT 10

Required configuration:

  <catboost_lib_path>/path/to/libcatboostmodel.so</catboost_lib_path>

*** Implementation Details ***

The internal protocol between the server and the library-bridge is
simple:

- HTTP GET on path "/extdict_ping":
  A ping, used during the handshake to check if the library-bridge runs.

- HTTP POST on path "extdict_request"
  (1) Send a "catboost_GetTreeCount" request from the server to the
      bridge, containing a library path (e.g /home/user/libcatboost.so) and
      a model path (e.g. /home/user/model.bin). Rirst, this unloads the
      catboost library handler associated to the model path (if it was
      loaded), then loads the catboost library handler associated to the
      model path, then executes GetTreeCount() on the library handler and
      finally sends the result back to the server. Step (1) is called once
      by the server from FunctionCatBoostEvaluate::getReturnTypeImpl(). The
      library path handler is unloaded in the beginning because it contains
      state which may no longer be valid if the user runs
      catboost("/path/to/model.bin", ...) more than once and if "model.bin"
      was updated in between.
  (2) Send "catboost_Evaluate" from the server to the bridge, containing
      the model path and the features to run the interference on. Step (2)
      is called multiple times (once per chunk) by the server from function
      FunctionCatBoostEvaluate::executeImpl(). The library handler for the
      given model path is expected to be already loaded by Step (1).

Fixes #27870
2022-09-08 09:01:32 +00:00
Alexey Milovidov
9544b8fdd6
Merge pull request #40996 from ClickHouse/vdimir/issue-40994
Minor update doc for mysql_port
2022-09-08 02:39:12 +03:00
Alexey Milovidov
84a00e3992
Merge pull request #41087 from peter279k/improve_clickhouse_start
Improve clickhouse start command
2022-09-08 02:35:02 +03:00
Kseniia Sumarokova
eb53df48d1
Update storing-data.md 2022-09-07 22:26:52 +02:00
Kseniia Sumarokova
3af51f4340
Update storing-data.md 2022-09-07 22:21:46 +02:00
Denny Crane
a75eb5ad84
Update date-time-functions.md 2022-09-07 15:59:23 -03:00