Commit Graph

84 Commits

Author SHA1 Message Date
Azat Khuzhin
238d44783b Fix flakiness of expect tests for clickhouse-client by avoiding history overlap
Yes, all writes to the history file is done under flock, *but*, before
writing the history file there is sort(), and so if you will run the
following tests the 01300_client_save_history_when_terminated_long will fail:

    $ /src/tests/clickhouse-test --print-time -j2 01180_client_syntax_errors 01300_client_save_history_when_terminated_long 0001_select

And it has nothing todo with timeouts:

    expect: does "" (spawn_id exp8) match glob pattern "for the history"? no
    f8f1dbfdaaca :) select (1, 2
    expect: does "\u001b[1Gf8f1dbfdaaca :) select \u001b[0;22;33m(\u001b[0;22;32m1\u001b[0;1m,\u001b[0m \u001b[0;22;32m2\u001b[0m\u001b[J" (spawn_id exp8) match glob pattern "for the history"? no

    expect: does "\u001b[1Gf8f1dbfdaaca :) select \u001b[0;22;33m(\u001b[0;22;32m1\u001b[0;1m,\u001b[0m \u001b[0;22;32m2\u001b[0m\u001b[J\u001b[29G" (spawn_id exp8) match glob pattern "for the history"? no
    expect: timed out

The "select (1, 2" is from 01180_client_syntax_errors

And use real file only when the history should be preserved across runs
(i.e. there are multiple invocations of clickhouse-client)

CI: https://s3.amazonaws.com/clickhouse-test-reports/0/1adfbac19fed9813725d8b1df14e617b58a45d20/stateless_tests__asan__[2/2].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-20 09:51:08 +01:00
Nikolay Degterinsky
bd4bc2df71
Update .gitignore 2023-01-18 01:49:52 +01:00
Azat Khuzhin
82aaad61aa Integrate skim into the client/local
Note, that it can the fail the client if the skim itself will fail,
however I haven't seen it panicd, so let's try.

P.S. about adding USE_SKIM into configure header instead of just compile
option for target, it is better, because it allows not to recompile lots
of C++ headers, since we have to add skim library as PUBLIC. But anyway
this will be resolved in a different way, but separatelly.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-14 20:57:41 +01:00
Azat Khuzhin
dab6919411 Add rust target folder to gitignore
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-14 20:57:22 +01:00
Azat Khuzhin
67fa185611
Revert "Builtin skim" 2022-12-14 17:17:19 +03:00
Alexey Milovidov
c93b7741b3
Merge branch 'master' into builtin-skim 2022-12-14 02:30:21 +03:00
Alexander Tokmakov
eaa7096c4e add *.generated-expect to gitignore 2022-12-12 14:05:37 +01:00
Azat Khuzhin
de58e9c02d Integrate skim into the client/local
Note, that it can the fail the client if the skim itself will fail,
however I haven't seen it panicd, so let's try.

P.S. about adding USE_SKIM into configure header instead of just compile
option for target, it is better, because it allows not to recompile lots
of C++ headers, since we have to add skim library as PUBLIC.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-11 15:52:00 +01:00
Azat Khuzhin
f8a070acbe Add rust target folder to gitignore
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-11 15:51:37 +01:00
Azat Khuzhin
0b48a2ed5e tests: add *.debuglog to gitignore
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-30 19:36:17 +01:00
Yatsishin Ilya
8e77180968 Merge remote-tracking branch 'origin/master' into revive-sqlancer 2022-10-26 08:31:00 +00:00
Yatsishin Ilya
3794d21089 ignore tests tmp files 2022-10-26 08:30:57 +00:00
Azat Khuzhin
5016597e58 Add arch packages to .gitignore
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-18 18:33:48 +03:00
Robert Schulze
60f9f6855d
feat: implement catboost in library-bridge
This commit moves the catboost model evaluation out of the server
process into the library-bridge binary. This serves two goals: On the
one hand, crashes / memory corruptions of the catboost library no longer
affect the server. On the other hand, we can forbid loading dynamic
libraries in the server (catboost was the last consumer of this
functionality), thus improving security.

SQL syntax:

  SELECT
    catboostEvaluate('/path/to/model.bin', FEAT_1, ..., FEAT_N) > 0 AS prediction,
    ACTION AS target
  FROM amazon_train
  LIMIT 10

Required configuration:

  <catboost_lib_path>/path/to/libcatboostmodel.so</catboost_lib_path>

*** Implementation Details ***

The internal protocol between the server and the library-bridge is
simple:

- HTTP GET on path "/extdict_ping":
  A ping, used during the handshake to check if the library-bridge runs.

- HTTP POST on path "extdict_request"
  (1) Send a "catboost_GetTreeCount" request from the server to the
      bridge, containing a library path (e.g /home/user/libcatboost.so) and
      a model path (e.g. /home/user/model.bin). Rirst, this unloads the
      catboost library handler associated to the model path (if it was
      loaded), then loads the catboost library handler associated to the
      model path, then executes GetTreeCount() on the library handler and
      finally sends the result back to the server. Step (1) is called once
      by the server from FunctionCatBoostEvaluate::getReturnTypeImpl(). The
      library path handler is unloaded in the beginning because it contains
      state which may no longer be valid if the user runs
      catboost("/path/to/model.bin", ...) more than once and if "model.bin"
      was updated in between.
  (2) Send "catboost_Evaluate" from the server to the bridge, containing
      the model path and the features to run the interference on. Step (2)
      is called multiple times (once per chunk) by the server from function
      FunctionCatBoostEvaluate::executeImpl(). The library handler for the
      given model path is expected to be already loaded by Step (1).

Fixes #27870
2022-09-08 09:01:32 +00:00
Robert Schulze
912663b719
Revert "Move CatBoost evaluation into clickhouse-library-bridge" 2022-08-31 20:54:43 +02:00
Robert Schulze
6b2b3c1eb3
feat: implement catboost in library-bridge
This commit moves the catboost model evaluation out of the server
process into the library-bridge binary. This serves two goals: On the
one hand, crashes / memory corruptions of the catboost library no longer
affect the server. On the other hand, we can forbid loading dynamic
libraries in the server (catboost was the last consumer of this
functionality), thus improving security.

SQL syntax:

  SELECT
    catboostEvaluate('/path/to/model.bin', FEAT_1, ..., FEAT_N) > 0 AS prediction,
    ACTION AS target
  FROM amazon_train
  LIMIT 10

Required configuration:

  <catboost_lib_path>/path/to/libcatboostmodel.so</catboost_lib_path>

*** Implementation Details ***

The internal protocol between the server and the library-bridge is
simple:

- HTTP GET on path "/extdict_ping":
  A ping, used during the handshake to check if the library-bridge runs.

- HTTP POST on path "extdict_request"
  (1) Send a "catboost_GetTreeCount" request from the server to the
      bridge, containing a library path (e.g /home/user/libcatboost.so) and
      a model path (e.g. /home/user/model.bin). Rirst, this unloads the
      catboost library handler associated to the model path (if it was
      loaded), then loads the catboost library handler associated to the
      model path, then executes GetTreeCount() on the library handler and
      finally sends the result back to the server. Step (1) is called once
      by the server from FunctionCatBoostEvaluate::getReturnTypeImpl(). The
      library path handler is unloaded in the beginning because it contains
      state which may no longer be valid if the user runs
      catboost("/path/to/model.bin", ...) more than once and if "model.bin"
      was updated in between.
  (2) Send "catboost_Evaluate" from the server to the bridge, containing
      the model path and the features to run the interference on. Step (2)
      is called multiple times (once per chunk) by the server from function
      FunctionCatBoostEvaluate::executeImpl(). The library handler for the
      given model path is expected to be already loaded by Step (1).

Fixes #27870
2022-08-29 20:26:45 +00:00
rfraposa
06ac99b1e7 Add cmake page back to docs && fix /settings/settings in /zh 2022-04-24 16:47:19 -06:00
Yatsishin Ilya
f59c7f5254 Add more build paths to .gitignore 2022-01-18 15:25:48 +00:00
Alexander Tokmakov
571dd3acfb fix style check 2021-09-21 10:28:33 +03:00
Azat Khuzhin
24c8968c80 Add *.log/*.stderr/*.stdout into gitignore 2021-06-08 09:14:47 +03:00
Nikolai Kochetov
77233bbdbb Ignore cmake-in-clickhouse 2021-04-28 18:10:12 +03:00
tison
c809af5dc2 ignore data store files 2021-02-17 12:58:17 +08:00
Ivan
315ff4f0d9
ANTLR4 Grammar for ClickHouse and new parser (#11298) 2020-12-04 05:15:44 +03:00
qianmoQ
e740bd40f1 fix document for index.md and distinctive-features.md 2020-11-24 20:36:19 +08:00
Ivan Lezhankin
7ea393ada8 Fix build without libraries 2020-10-10 23:41:27 +03:00
vladimir golovchenko
cb153d2605 Updated gitignore-files. 2020-08-06 18:05:32 -07:00
Ivan Blinkov
4ef322274d Add integration test 2020-05-29 22:53:16 +03:00
Alexey Milovidov
2986fcd93e Remove outdated contents from gitignore 2020-05-05 19:38:40 +03:00
Ivan Blinkov
98769778f4
Turkish docs translation stub (#10282) 2020-04-15 16:56:49 +03:00
Ivan Blinkov
765dd7c495
Update some docs translations (#10044) 2020-04-04 12:15:31 +03:00
Ivan Lezhankin
06446b4f08 dbms/ → src/ 2020-04-03 18:14:31 +03:00
Ivan
97f2a2213e
Move all folders inside /dbms one level up (#9974)
* Move some code outside dbms/src folder
* Fix paths
2020-04-02 02:51:21 +03:00
Ivan Blinkov
03aa7894d9
Draft of docs translation helper (#9755)
* replace exit with assert in test_single_page

* improve save_raw_single_page docs option

* More grammar fixes

* "Built from" link in new tab

* fix mistype

* Example of include in docs

* add anchor to meeting form

* Draft of translation helper

* WIP on translation helper

* Replace some fa docs content with machine translation
2020-03-19 20:49:27 +03:00
Ivan Blinkov
5abe3ac3f1
Switch docs to python3 and update MkDocs to 1.1 (#9711)
+ some grammar and css fixes
2020-03-18 16:02:32 +03:00
Ivan Blinkov
242a1a85d4 adjust .gitignore 2020-02-14 12:34:18 +03:00
Ivan Blinkov
18538f5c65 Domain change in docs 2020-01-30 13:34:55 +03:00
Ivan Lezhankin
e63ef08af8 Update gitignore 2020-01-14 16:30:06 +03:00
Ivan
4f2f5cca84
Add support for cross-compiling to the CPU architecture AARCH64 (#7370) 2019-10-30 10:01:53 +03:00
Alexander Tokmakov
93c672aa0b delete BlockInputStreamFromRowInputStream 2019-08-27 21:29:56 +03:00
Nikita Mikhaylov
b8f99255ae better gitignore with mrk2 2019-07-04 21:30:01 +03:00
Ivan Lezhankin
92769a2460 Don't update "arrow" on client-side for nothing 2019-02-07 16:47:16 +03:00
Alexander GQ Gerasiov
afa9e8d4ea .gitignore: Move debian/ specific entries to debian/.gitignore
Signed-off-by: Alexander GQ Gerasiov <gq@cs.msu.su>
2019-01-21 01:26:50 +03:00
Ivan Lezhankin
77daa519ff Update librdkafka to v1.0.0-RC5 2019-01-14 14:15:57 +03:00
Ivan Lezhankin
6df757c6f7 Refactor constant folding and make it reusable for primary_key_expr 2018-12-17 17:59:01 +03:00
Ivan Lezhankin
935615a647 Reimplement FREEZE command. 2018-11-12 15:26:14 +03:00
proller
a7437b93a9 Commited StorageSystemContributors.generated.cpp (#3510)
* CLICKHOUSE-4085 system.contributors

* fi

* Fix random

* Commited StorageSystemContributors.generated.cpp

* fix

* Update CMakeLists.txt
2018-11-01 17:05:37 +03:00
abyss7
d538f70679 Fix build and tests on Fedora (#3496)
* Fix some tests and build on Fedora 28

* Update contrib/ssl

* Try `sudo` first, then without `sudo`.
2018-10-30 17:05:44 +03:00
proller
c35c979285 CLICKHOUSE-4085 system.contributors (#3452)
* CLICKHOUSE-4085 system.contributors

* fi

* Fix random
2018-10-26 20:43:50 +03:00
Alexey Milovidov
3021852383 Imported stateful tests (without data) [#CLICKHOUSE-3] 2018-08-07 20:08:51 +03:00
Ivan Blinkov
b589903680
WIP on docs (#2753)
* Some improvements for introduction/performance.md

* Minor improvements for example_datasets

* Add website/package-lock.json to .gitignore

* YT paragraph was badly outdated and there is no real reason to write a new one

* Use weird introduction article as a starting point for F.A.Q.

* Some refactoring of first half of ya_metrika_task.md

* minor

* Weird docs footer bugfix
2018-07-30 19:34:55 +03:00