thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-29 11:02:08 +00:00
Code Issues Packages Projects Releases Wiki Activity
96,605 Commits 9,189 Branches 1,782 Tags 2.2 GiB
f53aa86a20
Commit Graph

20563 Commits

Author SHA1 Message Date
Sema Checherinda
e436b4f4cc Metric for the number of detached parts 2022-08-30 02:12:57 +02:00
alesapin
ac41de73df Add cache to zero copy replication test 2022-08-30 00:17:01 +02:00
Kseniia Sumarokova
8001560082
Fix test 2022-08-30 00:14:00 +02:00
Cory Levy
fe24bd9a7f Add tests 2022-08-29 16:47:33 -04:00
Robert Schulze
4ae7586a2d
fix: test 02415_all_new_functions_must_be_documented 
I am happy to move the documentation into the source code but currently
we'd need to maintain it twice (source code + traditional markdown docs)
because we don't render the docs from the source code.
 2022-08-29 20:27:06 +00:00
Robert Schulze
35a37c91f8
chore: incorporate review feedback 2022-08-29 20:27:06 +00:00
Robert Schulze
3aec76d281
fix: test 02117_show_create_table_system 2022-08-29 20:27:06 +00:00
robot-clickhouse
64fa077148
style: fix style 2022-08-29 20:27:06 +00:00
Robert Schulze
4d511332c4
chore: delete obsolete modelEvaluate() function 
- superseded by catboostEvaluate() which no longer uses the internal
  repository for external models

- also removed was statement SYSTEM RELOAD MODELS and the monitoring view
  SYSTEM.SYSTEMMODELS
 2022-08-29 20:27:06 +00:00
Robert Schulze
6b2b3c1eb3
feat: implement catboost in library-bridge 
This commit moves the catboost model evaluation out of the server
process into the library-bridge binary. This serves two goals: On the
one hand, crashes / memory corruptions of the catboost library no longer
affect the server. On the other hand, we can forbid loading dynamic
libraries in the server (catboost was the last consumer of this
functionality), thus improving security.

SQL syntax:

  SELECT
    catboostEvaluate('/path/to/model.bin', FEAT_1, ..., FEAT_N) > 0 AS prediction,
    ACTION AS target
  FROM amazon_train
  LIMIT 10

Required configuration:

  <catboost_lib_path>/path/to/libcatboostmodel.so</catboost_lib_path>

*** Implementation Details ***

The internal protocol between the server and the library-bridge is
simple:

- HTTP GET on path "/extdict_ping":
  A ping, used during the handshake to check if the library-bridge runs.

- HTTP POST on path "extdict_request"
  (1) Send a "catboost_GetTreeCount" request from the server to the
      bridge, containing a library path (e.g /home/user/libcatboost.so) and
      a model path (e.g. /home/user/model.bin). Rirst, this unloads the
      catboost library handler associated to the model path (if it was
      loaded), then loads the catboost library handler associated to the
      model path, then executes GetTreeCount() on the library handler and
      finally sends the result back to the server. Step (1) is called once
      by the server from FunctionCatBoostEvaluate::getReturnTypeImpl(). The
      library path handler is unloaded in the beginning because it contains
      state which may no longer be valid if the user runs
      catboost("/path/to/model.bin", ...) more than once and if "model.bin"
      was updated in between.
  (2) Send "catboost_Evaluate" from the server to the bridge, containing
      the model path and the features to run the interference on. Step (2)
      is called multiple times (once per chunk) by the server from function
      FunctionCatBoostEvaluate::executeImpl(). The library handler for the
      given model path is expected to be already loaded by Step (1).

Fixes #27870
 2022-08-29 20:26:45 +00:00
Vitaly Baranov
33f72fb011
Merge pull request #40060 from ClickHouse/vitlibar-increase-timeout-for-test_concurrent_backups 
Increase timeout for test_concurrent_backups
 2022-08-29 22:25:56 +02:00
Arthur Passos
dd49b44abb Fix host_regexp hosts file tst 2022-08-29 15:58:18 -03:00
Arthur Passos
961365c7a4 Fix CaresPTRResolver not reading hosts file 2022-08-29 15:11:39 -03:00
Kseniia Sumarokova
c5c48e44ea
Merge branch 'master' into fix-mysql-timeouts 2022-08-29 19:33:29 +02:00
Filatenkov Artur
d73f661732
Merge branch 'master' into annoy-2 2022-08-29 17:33:13 +03:00
Anton Popov
506bee1e2f avoid logical errors at some common operation with ColumnObject 2022-08-29 14:31:09 +00:00
kssenii
0a6c4b9265 Fix 2022-08-29 16:20:53 +02:00
Vladimir C
5cbe7e0846
Merge pull request #40548 from ClickHouse/vdimir/warn-suppress-40330 
Add config option warning_supress_regexp
 2022-08-29 14:02:00 +02:00
Alexander Tokmakov
8d90d30d37
Merge pull request #40589 from ClickHouse/remove_wrong_code_from_mutations 
Remove wrong code for skipping mutations in MergeTree
 2022-08-29 14:18:59 +03:00
avogar
85fa372ad7 Fix possible error Attempt to read after eof in CSV schema inference 2022-08-29 11:18:53 +00:00
Anton Popov
2a3e012931
Merge branch 'master' into fix-read-in-order-fixed-prefix 2022-08-29 13:17:26 +02:00
Alexander Tokmakov
eda0582ec0
Merge pull request #40641 from ClickHouse/fix_startup_of_dropped_replica 
Do not try to strartup dropped replica
 2022-08-29 14:15:15 +03:00
Vitaly Baranov
2bec3d3a7c Increase timeout for test_concurrent_backups 2022-08-29 13:13:43 +02:00
alesapin
7ce0afc0df
Merge pull request #40670 from Avogar/kafka 
Add setting to disable limit on kafka_num_consumers
 2022-08-29 10:53:35 +02:00
Amos Bird
15a69bce84
Use index when row_policy_filter is always false 2022-08-29 16:44:32 +08:00
Alexey Milovidov
b72fceb441
Merge pull request #40708 from lesandie/test_s3_table_functions 
Added integration test for s3 table function
 2022-08-29 03:49:53 +03:00
Alexey Milovidov
18eaf7d0dc
Merge pull request #40721 from ClickHouse/enable_zero_copy_replication_in_ci 
Enable zero-copy replication in CI
 2022-08-29 03:49:23 +03:00
Alexey Milovidov
71f6c52c2d
Merge pull request #40727 from amosbird/column-transformer-fix1 
Correct format of APPLY transformer param
 2022-08-29 03:48:04 +03:00
Alexey Milovidov
82ef85e713
Merge pull request #40722 from kssenii/fix-test-02382 
Fix flaky test
 2022-08-28 23:34:10 +03:00
Diego Nieto (lesandie)
3d50dbea34 Black reformatting 2022-08-28 20:05:39 +02:00
Alexey Milovidov
a82723b5d9
Merge pull request #40719 from ClickHouse/kssenii-patch-4 
Update 02313_filesystem_cache_seeks.queries
 2022-08-28 21:04:48 +03:00
Amos Bird
d1fbe51b81
Correct format of APPLY transformer param 2022-08-29 01:21:12 +08:00
alesapin
bcc8106182 Merge branch 'master' into enable_zero_copy_replication_in_ci 2022-08-28 17:04:08 +02:00
alesapin
98d84402e6 Fix test 2022-08-28 17:03:04 +02:00
Alexey Milovidov
ddff5e9145
Merge pull request #40712 from ClickHouse/ci-cmake-self-extractable 
cmake: Disable default ENABLE_CLICKHOUSE_SELF_EXTRACTING
 2022-08-28 17:12:19 +03:00
Alexey Milovidov
fc29303ff3
Merge pull request #40685 from ClickHouse/nlp-functional-test 
Rewrite NLP tests from integration to functional
 2022-08-28 17:05:42 +03:00
Alexey Milovidov
f518844f06
Merge pull request #40709 from ClickHouse/all_new_table_functions_must_be_documented 
All new table functions must be documented
 2022-08-28 17:05:20 +03:00
Alexey Milovidov
00074a5e14
Merge pull request #40711 from ClickHouse/all_new_functions_must_be_documented 
All new functions must be documented
 2022-08-28 17:05:05 +03:00
kssenii
3be79fc145 Fix 2022-08-28 15:57:46 +02:00
alesapin
0b827207db Enable zero-copy replication in CI 2022-08-28 15:46:11 +02:00
Kseniia Sumarokova
d6efbf039e
Update 02313_filesystem_cache_seeks.queries 2022-08-28 14:49:53 +02:00
Kseniia Sumarokova
a0bc5b6ea4
Merge pull request #40705 from azat/stress/max_read_buffer_size 
Fix LOGICAL_ERROR with max_read_buffer_size=0 during reading marks
 2022-08-28 12:41:16 +02:00
Robert Schulze
df934d8762
Merge pull request #40217 from zvonand/zvonand-minmax 
Fix conversion Date32 / DateTime64 / Date to narrow types
 2022-08-28 09:42:39 +02:00
Alexey Milovidov
18c56b722f Fix integration test 2022-08-28 04:30:27 +02:00
Alexey Milovidov
d3a7945b1b
Merge pull request #40646 from ClickHouse/force-documentation-3 
Every profile event must have documentation
 2022-08-28 04:10:11 +03:00
Alexey Milovidov
632867dbff Fix error 2022-08-28 02:40:43 +02:00
Alexey Milovidov
0f59e0511b
Merge pull request #40703 from ClickHouse/fix_02242_system_filesystem 
Make 02242_system_filesystem_cache_log_table output stable
 2022-08-28 03:25:44 +03:00
Alexey Milovidov
ccfce4f51a Update test 2022-08-28 02:10:04 +02:00
Alexey Milovidov
5b57c91ad6 Rename table function MeiliSearch to meilisearch 2022-08-28 02:08:51 +02:00
Alexey Milovidov
1a7dbd2c62 Update test 2022-08-28 02:05:58 +02:00
Alexey Milovidov
7831200fc8 All new table functions must be documented 2022-08-28 02:05:58 +02:00
Alexey Milovidov
b0e5174e78 Update test 2022-08-28 02:01:17 +02:00
Alexey Milovidov
0309a2ca76 Fix test 2022-08-28 01:37:23 +02:00
Alexey Milovidov
d583f29a00 Fix test 2022-08-28 01:36:32 +02:00
Alexey Milovidov
63fc3a2c3f Update test 2022-08-28 00:27:37 +02:00
Alexey Milovidov
89daf3db8f Fix test 2022-08-28 00:18:00 +02:00
Alexey Milovidov
6ac9eabbd4 Register more aliases correctly 2022-08-28 00:17:13 +02:00
Alexey Milovidov
a6d99e795e Register more aliases correctly 2022-08-28 00:14:38 +02:00
Alexey Milovidov
a7f4c90f0c Update test 2022-08-27 23:59:32 +02:00
Alexander Gololobov
f90033d174 Make 02242_system_filesystem_cache_log_table output stable 2022-08-27 23:33:06 +02:00
Yakov Olkhovskiy
31a7ed09a1 disable default ENABLE_CLICKHOUSE_SELF_EXTRACTING and add to env 2022-08-27 21:08:01 +00:00
Alexey Milovidov
7bd1142f63
Merge pull request #40647 from ClickHouse/high-level-coverage 
Report high-level function and data types test coverage
 2022-08-27 23:13:10 +03:00
Alexey Milovidov
6139cc8f7a Add Documentation to FunctionFactory 2022-08-27 22:06:57 +02:00
Alexey Milovidov
f86242c17e
Merge pull request #40699 from ClickHouse/remove-unbundled 
Remove the remaining parts of "unbundled" build
 2022-08-27 21:54:42 +03:00
Alexey Milovidov
e7054debc1
Merge pull request #40649 from ClickHouse/embedded-documentation 
A prototype for embedded documentation
 2022-08-27 21:28:11 +03:00
Alexey Milovidov
ef191efdc8 Merge branch 'nlp-functional-test' of github.com:ClickHouse/ClickHouse into nlp-functional-test 2022-08-27 20:23:31 +02:00
Alexey Milovidov
b70373d429 Fix error 2022-08-27 20:23:15 +02:00
Alexey Milovidov
00afbadb53
Merge branch 'master' into high-level-coverage 2022-08-27 19:34:26 +03:00
Diego Nieto (lesandie)
9a07489151 Added integration test for s3 table function 2022-08-27 18:33:04 +02:00
Alexey Milovidov
a6387ac0f7
Merge branch 'master' into force-documentation-3 2022-08-27 19:31:44 +03:00
Alexey Milovidov
6e564b18bf
Merge pull request #40600 from FrankChen021/check_url_arg 
Validate the CompressionMethod parameter of URL table engine
 2022-08-27 19:29:55 +03:00
Alexey Milovidov
7569cbb381
Merge pull request #40643 from ClickHouse/fix-recursive-buffer 
Fix stack overflow in recursive Buffer tables.
 2022-08-27 19:27:24 +03:00
Alexey Milovidov
e358d57184 Check for coverage 2022-08-27 18:11:16 +02:00
Alexey Milovidov
abfddf17e2
Merge pull request #40658 from ClickHouse/vdimir/perf-test-report-env 
Fix env variable forwaring in performance_comparison_check.py
 2022-08-27 17:01:57 +03:00
Alexey Milovidov
b4cf54e3ef
Merge pull request #40683 from ClickHouse/test-to-decimal-or-default 
Add a test for toDecimal*OrDefault functions
 2022-08-27 16:53:55 +03:00
Alexey Milovidov
36ca126a26
Merge pull request #40679 from ClickHouse/fix-typo 
Fix typo
 2022-08-27 16:52:21 +03:00
Alexey Milovidov
fef269d0b8
Merge pull request #40686 from ClickHouse/model-evaluate-smoke-test 
Add a smoke (fake) test for `modelEvaluate` function
 2022-08-27 16:52:09 +03:00
Azat Khuzhin
c801469e9e Fix LOGICAL_ERROR with max_read_buffer_size=0 during reading marks 
The problem is that the buffer size validated after marks reading in
MergeTreeReaderStream::init(), since it requires to read marks first.

And later it is passed to AsynchronousReadBufferFromFileDescriptor,
which throws LOGICAL_ERROR because buffer_size < alignment.

Fix this my simply disallow such values for max_read_buffer_size (I
thougt about modifying createReadBufferFromFileBase(), but it is not
used for remote reads -- remote_fs_buffer_size).

Fixes: #40669
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-27 14:12:07 +02:00
Alexander Gololobov
6fc8f1e9e0
Merge pull request #40645 from ClickHouse/force-documentation-2 
Every current metric must have documentation
 2022-08-27 10:44:47 +02:00
Duc Canh Le
1691db9ce6
Merge branch 'master' into ch_canh_fix_json 2022-08-27 09:23:04 +08:00
Alexey Milovidov
fc3514ca1a Maybe better 2022-08-27 03:13:53 +02:00
Alexey Milovidov
91ed3ebe9a Remove the remaining parts of "unbundled" build 2022-08-27 02:18:47 +02:00
Alexey Milovidov
66bee79d8b
Update install.sh 2022-08-27 02:56:56 +03:00
Alexey Milovidov
04dc970cb8 Even more reports 2022-08-26 23:57:14 +02:00
Alexey Milovidov
2a7d9f0778 Merge branch 'master' into high-level-coverage 2022-08-26 23:44:52 +02:00
Alexey Milovidov
86fc10af66 Add missing file 2022-08-26 23:42:36 +02:00
Alexey Milovidov
8000e9ec92 Fix test 2022-08-26 23:09:01 +02:00
Alexey Milovidov
2e16e2e9ee Fix fasttest 2022-08-26 22:43:15 +02:00
Alexey Milovidov
3f0ac45389 Add a smoke (fake) test for modelEvaluate function 2022-08-26 22:40:06 +02:00
Alexey Milovidov
6414a01987 Rewrite NLP tests from integration to functional 2022-08-26 22:36:34 +02:00
Alexey Milovidov
2eeedb345e Add a test for toDecimal*OrDefault functions 2022-08-26 21:42:43 +02:00
Alexander Gololobov
4fef408104
Merge branch 'master' into force-documentation-3 2022-08-26 20:29:06 +02:00
Alexey Milovidov
6940bdab3b
Merge branch 'master' into fix-recursive-buffer 2022-08-26 21:25:11 +03:00
Alexander Tokmakov
91ec3793db
Merge pull request #40665 from ClickHouse/vdimir/02177_issue_31009_release_only 
Disable 02177_issue_31009 for non-release builds
 2022-08-26 21:19:37 +03:00
Alexander Tokmakov
3868a029f7
Merge pull request #40674 from ClickHouse/tavplubix-patch-4 
Make 01111_create_drop_replicated_db_stress less flaky
 2022-08-26 21:15:06 +03:00
Alexey Milovidov
1ad44acce8
Merge pull request #40644 from ClickHouse/force-documentation 
Every setting must have documentation
 2022-08-26 21:06:57 +03:00
zvonand
b9b8f7a05d explicit timezone in test 2022-08-26 18:43:31 +03:00
Kruglov Pavel
d4d0ec1c95
Fix style 2022-08-26 16:44:54 +02:00
Alexander Tokmakov
407bfdb568
Update 01111_create_drop_replicated_db_stress.sh 2022-08-26 16:59:40 +03:00
FArthur-cmd
f3d0083e57 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into annoy-2 2022-08-26 13:53:53 +00:00