Commit Graph

20771 Commits

Author SHA1 Message Date
alesapin
521e2cc22e Merge branch 'master' into stress_s3 2022-09-02 12:57:29 +02:00
Lloyd-Pottiger
f499381f57 move functional test to integration test
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
2022-09-02 18:45:37 +08:00
Lloyd-Pottiger
5a6b2106b5 Merge branch 'master' of github.com:ClickHouse/ClickHouse into feat/support-read-only-for-embeddedrocksdb 2022-09-02 18:10:00 +08:00
Alexander Tokmakov
71dc70bdff
Merge pull request #40876 from ClickHouse/tavplubix-patch-3
"Fix" database names collision in functional tests
2022-09-02 12:24:23 +03:00
Robert Schulze
a0544b2a01
test: Add test for '.' and '/' repeat commands in client interactive mode 2022-09-02 08:01:45 +00:00
Robert Schulze
319d8b00a7
Merge pull request #39010 from FrankChen021/tracing_context_propagation
Improve the opentelemetry tracing context propagation across threads
2022-09-02 07:56:43 +02:00
Robert Schulze
c7c00f9002
Merge pull request #40739 from ClickHouse/clang-tidy-for-headers
Enable clang-tidy for headers
2022-09-02 07:54:50 +02:00
Duc Canh Le
763328ef53 Merge branch 'master' into ch_canh_fix_set 2022-09-02 11:45:13 +08:00
Lloyd-Pottiger
59dccd6e49 fix test
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
2022-09-02 11:14:49 +08:00
Kruglov Pavel
5a9d2460dd
Merge pull request #40851 from Avogar/fix-json-object-inference
Fix nested JSON Objects schema inference
2022-09-01 21:07:21 +02:00
Mikhail f. Shiryaev
9e4f5d5006
Merge pull request #40845 from ClickHouse/cherry-pick-assignees
Assign new cherry-pick and backport PRs
2022-09-01 18:32:46 +02:00
Dmitry Novik
ddadb362cf
Merge pull request #39762 from quickhouse/betterorderbyoptimization
Fixed `Unknown identifier (aggregate-function)` exception which appears when a user tries to calculate WINDOW ORDER BY/PARTITION BY expressions over aggregate functions
2022-09-01 18:08:06 +02:00
Dmitry Novik
2ca619d40f
Merge pull request #40320 from azat/memory-tracker-fix-peak-race
Fix rare race in updating peak of memory tracker
2022-09-01 18:07:33 +02:00
Frank Chen
9d63cbe811 Merge 'origin/master' into tracing_context_propagation to resolve conflicts 2022-09-01 23:18:59 +08:00
alesapin
ad69bfcd6f Merge branch 'stress_s3' of github.com:ClickHouse/ClickHouse into stress_s3 2022-09-01 17:05:55 +02:00
alesapin
89930c315b Merge branch 'master' into stress_s3 2022-09-01 17:03:55 +02:00
Robert Schulze
f1c0501631
Merge pull request #40847 from ClickHouse/sanitizer-naming
Consistent sanitizer naming in CI
2022-09-01 16:42:55 +02:00
Antonio Andelic
a0c1ce5eb7 Merge branch 'master' into embeddedrocksdb-delete-update-support 2022-09-01 14:40:24 +00:00
Nikolay Degterinsky
5a611e272a Merge remote-tracking branch 'upstream/master' into obfuscator-schema-inference 2022-09-01 13:07:28 +00:00
Vladimir C
12e6fc4182
Merge branch 'master' into join_and_sets 2022-09-01 14:56:14 +02:00
Vladimir C
f2cf7d7762
Merge pull request #40334 from ClickHouse/vdimir/analyze-stuck-limit
Limit number of analyze for one query, att. 2
2022-09-01 14:40:20 +02:00
Kseniia Sumarokova
c6c67a248d
Merge pull request #40792 from canhld94/ch_canh_intersect_distinct
Implement intersect + except distinct
2022-09-01 14:35:26 +02:00
Fangyuan Deng
bc7d661668
Merge branch 'master' into UniqSketch 2022-09-01 19:31:53 +08:00
Alexander Tokmakov
ecdea62cd5
Update clickhouse-test 2022-09-01 14:26:50 +03:00
alesapin
4058db5cba
Merge branch 'master' into stress_s3 2022-09-01 13:22:55 +02:00
Vitaly Baranov
6dcca686cb
Merge pull request #40827 from vitlibar/fix-incremental-backups-for-log-family
Fix incremental backups for Log family.
2022-09-01 13:13:32 +02:00
Mikhail f. Shiryaev
ca837e6755
Add a doc-string for ReleaseBranch._assign_new_pr 2022-09-01 13:06:14 +02:00
Anton Popov
f7bdf07adc
Merge pull request #38715 from CurtizJ/fix-read-in-order-fixed-prefix
Better support of `optimize_read_in_order` in case of fixed prefix of sorting key
2022-09-01 12:59:18 +02:00
Anton Popov
cc49198204
Merge pull request #40754 from CurtizJ/fix-fuzzer-with-object
Avoid logical errors in some common operations with `ColumnObject`
2022-09-01 12:57:32 +02:00
Mikhail f. Shiryaev
1da02adc8d
Assign PRs to an original merged_by too 2022-09-01 12:38:30 +02:00
Sema Checherinda
7b59fdc042
Merge pull request #40779 from CheSema/detached-parts-metric
Metric for the number of detached parts
2022-09-01 12:24:42 +02:00
Duc Canh Le
018f4118bb
Fix view with merge table function (#40734) 2022-09-01 12:20:00 +02:00
luocongkai
56e4179cad fix bug when remove unneeded columns in subquery 2022-09-01 17:10:32 +08:00
Robert Schulze
de64c6b103
Merge branch 'master' into clang-tidy-for-headers 2022-09-01 10:24:56 +02:00
Robert Schulze
d311482391
More renamings for consistency
AST Fuzzer (ASan)  --> AST Fuzzer (asan)
AST Fuzzer (TSan)  --> AST Fuzzer (tsan)
AST Fuzzer (MSan)  --> AST Fuzzer (msan)
AST Fuzzer (UBSan) --> AST Fuzzer (ubsan)
2022-09-01 07:59:01 +00:00
Robert Schulze
56eece40ec
Merge pull request #40736 from LevyCory/add-offset-to-formatDateTime
Add timezone offset support  to `formatDateTime`
2022-09-01 09:50:17 +02:00
Nikolay Degterinsky
e98ceb2575
Merge pull request #40251 from ClibMouse/Structured-logging-custom-keys
Feature Improvement: Custom Key Names for Structured Logging Support
2022-09-01 00:36:23 +02:00
Robert Schulze
912663b719
Revert "Move CatBoost evaluation into clickhouse-library-bridge" 2022-08-31 20:54:43 +02:00
Alexander Tokmakov
e659b4553d
Update clickhouse-test 2022-08-31 21:53:57 +03:00
Kruglov Pavel
86516d3bb4
Merge pull request #40740 from amosbird/row-policy-index-fix-1
Use index when row_policy_filter is always false
2022-08-31 18:46:14 +02:00
Kruglov Pavel
b7ea529307
Merge pull request #40746 from Avogar/fix-csv-schema-inference
Fix possible error Attempt to read after eof in CSV schema inference
2022-08-31 18:29:01 +02:00
Antonio Andelic
f043790e6d Add support for update 2022-08-31 14:33:53 +00:00
Robert Schulze
ca01286028
Merge pull request #39629 from ClickHouse/catboost-bridge
Move CatBoost evaluation into clickhouse-library-bridge
2022-08-31 16:16:11 +02:00
avogar
53ebe8a0bd Add test 2022-08-31 14:11:00 +00:00
Anton Popov
5a3e24c4e4
Merge pull request #40483 from canhld94/ch_canh_fix_json
Fix columnObject crash with array of variadic dimension elems
2022-08-31 16:07:58 +02:00
Duc Canh Le
55e675a407 fix test again 2022-08-31 21:57:07 +08:00
Duc Canh Le
5f1827a316 fix style test 2022-08-31 21:52:57 +08:00
Duc Canh Le
55c20ce45b fix tryGetPreparedSet crash when same set expression built from different columns 2022-08-31 21:43:08 +08:00
Antonio Andelic
0e6b3b870a Add support for delete in RocksDB 2022-08-31 13:08:27 +00:00
alesapin
b8895bb01b Merge branch 'fix_incorrect_remove_in_zero_copy' into stress_s3 2022-08-31 14:55:07 +02:00
Robert Schulze
48164b039a
Consistent sanitizer naming in CI
Sanitizer build/test jobs were sometimes named with full form
('undefined') - which could be confusing - and sometimes named with
abbreviated form ('ubsan'). Now always using the short form.
2022-08-31 12:45:05 +00:00
Mikhail f. Shiryaev
187b10dec5
Assign to all at once, improve logging 2022-08-31 14:09:17 +02:00
Vitaly Baranov
007ae0e6cc Fix incremental backups for Log family. 2022-08-31 12:57:28 +02:00
Robert Schulze
cedf75ed5e
Enable clang-tidy for headers
clang-tidy now also checks code in header files. Because the analyzer
finds tons of issues, activate the check only for directory "base/" (see
file ".clang-tidy"). All other directories, in particular "src/" are
left to future work.

While many findings were fixed, some were not (and suppressed instead).
Reasons for this include: a) the file is 1:1 copypaste of a 3rd-party
lib (e.g. pcg_extras.h) and fixing stuff would make upgrades/fixes more
difficult b) a fix would have broken lots of using code
2022-08-31 10:48:15 +00:00
vdimir
7d8e0250d9 Mark 02337_join_analyze_stuck long 2022-08-31 11:57:41 +02:00
vdimir
33508507bf Upd 02337_join_analyze_stuck 2022-08-31 11:57:41 +02:00
vdimir
f38f39ed6f Use max_analyze_depth 2022-08-31 11:57:41 +02:00
vdimir
928adb2b38 [WIP] Limit number of analyze for one query 2022-08-31 11:57:41 +02:00
Duc Canh Le
aebd1cb7de
Merge branch 'master' into ch_canh_intersect_distinct 2022-08-31 12:08:20 +08:00
Fangyuan Deng
5c2f35c302
Merge branch 'master' into UniqSketch 2022-08-31 10:23:46 +08:00
root
ea517de61a style check correction 2022-08-30 23:20:19 -03:00
root
f02f7ce99c addressed minor change requests 2022-08-30 23:20:19 -03:00
root
84a617675d style check correction in test.py 2022-08-30 23:20:19 -03:00
root
3ced3c9933 style check correction in test.py 2022-08-30 23:20:19 -03:00
root
1d29a494db Modified test code to cover various scenarios for custom keys 2022-08-30 23:20:19 -03:00
root
d5db88fa27 Structured-logging-custom-keys PR init 2022-08-30 23:20:19 -03:00
Anton Popov
3504781529
Merge branch 'master' into fix-read-in-order-fixed-prefix 2022-08-30 23:32:43 +02:00
Sema Checherinda
9bb4f556c3 work on remarks on the review 2022-08-30 20:18:35 +02:00
Cory Levy
4a9ccc6496 Fix timezone offset tests 2022-08-30 13:42:12 -04:00
alesapin
216773fc47 Merge branch 'stress_s3' of github.com:ClickHouse/ClickHouse into stress_s3 2022-08-30 18:45:12 +02:00
alesapin
5a0c5f0542 Merge branch 'kssenii-fix-write-through-cache-logical-error' into stress_s3 2022-08-30 18:44:14 +02:00
Alexander Tokmakov
6a50c20734
Merge branch 'master' into zookeeper_client_fault_injection 2022-08-30 15:43:30 +03:00
Dmitry Novik
0a8378d9cd
Merge branch 'master' into betterorderbyoptimization 2022-08-30 14:23:22 +02:00
Alexander Tokmakov
6fdfb964d0
Revert "Add Annoy index" 2022-08-30 15:10:10 +03:00
vdimir
0f6f3c73b0
Minor fix 2022-08-30 11:57:28 +00:00
Vladimir C
7266c007b7
Merge pull request #39131 from amosbird/cte-global-with-fix1
Fix unused unknown columns introduced by WITH statement
2022-08-30 13:52:19 +02:00
Anton Popov
745a068959
Update 01825_type_json_order_by.sql 2022-08-30 13:41:48 +02:00
Alexander Tokmakov
022f440ad0
Merge pull request #40769 from arthurpassos/caresptrresolver-hosts-patch
Fix CaresPTRResolver not reading hosts file
2022-08-30 14:35:10 +03:00
alesapin
2588901bc9
Merge pull request #40780 from ClickHouse/add_cache_to_zero_copy_test
Add cache to zero copy replication test
2022-08-30 12:42:05 +02:00
Duc Canh Le
626b3e4147
Merge branch 'master' into ch_canh_intersect_distinct 2022-08-30 18:14:07 +08:00
Duc Canh Le
8590cc46c4 implement intersect + except distinct 2022-08-30 18:09:01 +08:00
Kseniia Sumarokova
c88db2ef97
Merge pull request #40751 from kssenii/fix-mysql-timeouts
Fix issue with mysql db / table function timeouts
2022-08-30 11:59:01 +02:00
Robert Schulze
cc4225109f
Merge pull request #37215 from Vector-Similarity-Search-for-ClickHouse/annoy-2
Test failures are unrelated, merging.
2022-08-30 09:25:57 +02:00
Robert Schulze
8ba9f6dd5d
fix: delete obsolete test 02413_model_evaluate_smoke 2022-08-30 07:21:05 +00:00
Frank Chen
f17d56b528 Merge branch 'master' into tracing_context_propagation 2022-08-30 14:24:36 +08:00
Cory Levy
cd371110ad Fix timezone offset formatting 2022-08-29 21:01:27 -04:00
Sema Checherinda
e436b4f4cc Metric for the number of detached parts 2022-08-30 02:12:57 +02:00
alesapin
ac41de73df Add cache to zero copy replication test 2022-08-30 00:17:01 +02:00
Kseniia Sumarokova
8001560082
Fix test 2022-08-30 00:14:00 +02:00
Cory Levy
fe24bd9a7f Add tests 2022-08-29 16:47:33 -04:00
Robert Schulze
4ae7586a2d
fix: test 02415_all_new_functions_must_be_documented
I am happy to move the documentation into the source code but currently
we'd need to maintain it twice (source code + traditional markdown docs)
because we don't render the docs from the source code.
2022-08-29 20:27:06 +00:00
Robert Schulze
35a37c91f8
chore: incorporate review feedback 2022-08-29 20:27:06 +00:00
Robert Schulze
3aec76d281
fix: test 02117_show_create_table_system 2022-08-29 20:27:06 +00:00
robot-clickhouse
64fa077148
style: fix style 2022-08-29 20:27:06 +00:00
Robert Schulze
4d511332c4
chore: delete obsolete modelEvaluate() function
- superseded by catboostEvaluate() which no longer uses the internal
  repository for external models

- also removed was statement SYSTEM RELOAD MODELS and the monitoring view
  SYSTEM.SYSTEMMODELS
2022-08-29 20:27:06 +00:00
Robert Schulze
6b2b3c1eb3
feat: implement catboost in library-bridge
This commit moves the catboost model evaluation out of the server
process into the library-bridge binary. This serves two goals: On the
one hand, crashes / memory corruptions of the catboost library no longer
affect the server. On the other hand, we can forbid loading dynamic
libraries in the server (catboost was the last consumer of this
functionality), thus improving security.

SQL syntax:

  SELECT
    catboostEvaluate('/path/to/model.bin', FEAT_1, ..., FEAT_N) > 0 AS prediction,
    ACTION AS target
  FROM amazon_train
  LIMIT 10

Required configuration:

  <catboost_lib_path>/path/to/libcatboostmodel.so</catboost_lib_path>

*** Implementation Details ***

The internal protocol between the server and the library-bridge is
simple:

- HTTP GET on path "/extdict_ping":
  A ping, used during the handshake to check if the library-bridge runs.

- HTTP POST on path "extdict_request"
  (1) Send a "catboost_GetTreeCount" request from the server to the
      bridge, containing a library path (e.g /home/user/libcatboost.so) and
      a model path (e.g. /home/user/model.bin). Rirst, this unloads the
      catboost library handler associated to the model path (if it was
      loaded), then loads the catboost library handler associated to the
      model path, then executes GetTreeCount() on the library handler and
      finally sends the result back to the server. Step (1) is called once
      by the server from FunctionCatBoostEvaluate::getReturnTypeImpl(). The
      library path handler is unloaded in the beginning because it contains
      state which may no longer be valid if the user runs
      catboost("/path/to/model.bin", ...) more than once and if "model.bin"
      was updated in between.
  (2) Send "catboost_Evaluate" from the server to the bridge, containing
      the model path and the features to run the interference on. Step (2)
      is called multiple times (once per chunk) by the server from function
      FunctionCatBoostEvaluate::executeImpl(). The library handler for the
      given model path is expected to be already loaded by Step (1).

Fixes #27870
2022-08-29 20:26:45 +00:00
Vitaly Baranov
33f72fb011
Merge pull request #40060 from ClickHouse/vitlibar-increase-timeout-for-test_concurrent_backups
Increase timeout for test_concurrent_backups
2022-08-29 22:25:56 +02:00
Arthur Passos
dd49b44abb Fix host_regexp hosts file tst 2022-08-29 15:58:18 -03:00
Arthur Passos
961365c7a4 Fix CaresPTRResolver not reading hosts file 2022-08-29 15:11:39 -03:00
Kseniia Sumarokova
c5c48e44ea
Merge branch 'master' into fix-mysql-timeouts 2022-08-29 19:33:29 +02:00
Filatenkov Artur
d73f661732
Merge branch 'master' into annoy-2 2022-08-29 17:33:13 +03:00
Anton Popov
506bee1e2f avoid logical errors at some common operation with ColumnObject 2022-08-29 14:31:09 +00:00
kssenii
0a6c4b9265 Fix 2022-08-29 16:20:53 +02:00
Vladimir C
5cbe7e0846
Merge pull request #40548 from ClickHouse/vdimir/warn-suppress-40330
Add config option warning_supress_regexp
2022-08-29 14:02:00 +02:00
Alexander Tokmakov
8bdb589c2b Merge branch 'master' into zookeeper_client_fault_injection 2022-08-29 13:34:57 +02:00
Alexander Tokmakov
8d90d30d37
Merge pull request #40589 from ClickHouse/remove_wrong_code_from_mutations
Remove wrong code for skipping mutations in MergeTree
2022-08-29 14:18:59 +03:00
avogar
85fa372ad7 Fix possible error Attempt to read after eof in CSV schema inference 2022-08-29 11:18:53 +00:00
Anton Popov
2a3e012931
Merge branch 'master' into fix-read-in-order-fixed-prefix 2022-08-29 13:17:26 +02:00
Alexander Tokmakov
eda0582ec0
Merge pull request #40641 from ClickHouse/fix_startup_of_dropped_replica
Do not try to strartup dropped replica
2022-08-29 14:15:15 +03:00
Vitaly Baranov
2bec3d3a7c Increase timeout for test_concurrent_backups 2022-08-29 13:13:43 +02:00
vdimir
afb6b7d9cf
Test plan and pipeline for filtering step for join 2022-08-29 09:49:15 +00:00
vdimir
2deb28d56e
Add 02382_join_and_filtering_set 2022-08-29 09:49:14 +00:00
vdimir
e21763e759
remove new setting from join_set_filter.xml 2022-08-29 09:49:13 +00:00
vdimir
470dcff89c
Add tests/performance/join_set_filter.xml 2022-08-29 09:49:11 +00:00
alesapin
e0fab1fa11
Merge branch 'master' into stress_s3 2022-08-29 11:37:29 +02:00
alesapin
40fb163e8c
Update s3_cache.xml 2022-08-29 11:37:19 +02:00
alesapin
133ca01447 Merge branch 'master' into stress_s3 2022-08-29 11:25:28 +02:00
alesapin
7ce0afc0df
Merge pull request #40670 from Avogar/kafka
Add setting to disable limit on kafka_num_consumers
2022-08-29 10:53:35 +02:00
Amos Bird
15a69bce84
Use index when row_policy_filter is always false 2022-08-29 16:44:32 +08:00
Amos Bird
6493fb4cbb
Add more test cases 2022-08-29 14:35:49 +08:00
Amos Bird
abf51bccfd
Disable projection when grouping set is used. 2022-08-29 14:35:48 +08:00
Alexey Milovidov
b72fceb441
Merge pull request #40708 from lesandie/test_s3_table_functions
Added integration test for s3 table function
2022-08-29 03:49:53 +03:00
Alexey Milovidov
18eaf7d0dc
Merge pull request #40721 from ClickHouse/enable_zero_copy_replication_in_ci
Enable zero-copy replication in CI
2022-08-29 03:49:23 +03:00
Alexey Milovidov
71f6c52c2d
Merge pull request #40727 from amosbird/column-transformer-fix1
Correct format of APPLY transformer param
2022-08-29 03:48:04 +03:00
Alexey Milovidov
82ef85e713
Merge pull request #40722 from kssenii/fix-test-02382
Fix flaky test
2022-08-28 23:34:10 +03:00
Diego Nieto (lesandie)
3d50dbea34 Black reformatting 2022-08-28 20:05:39 +02:00
Alexey Milovidov
a82723b5d9
Merge pull request #40719 from ClickHouse/kssenii-patch-4
Update 02313_filesystem_cache_seeks.queries
2022-08-28 21:04:48 +03:00
Amos Bird
d1fbe51b81
Correct format of APPLY transformer param 2022-08-29 01:21:12 +08:00
alesapin
cdc7f63f4f
Merge branch 'master' into stress_s3 2022-08-28 17:07:28 +02:00
alesapin
bcc8106182 Merge branch 'master' into enable_zero_copy_replication_in_ci 2022-08-28 17:04:08 +02:00
alesapin
98d84402e6 Fix test 2022-08-28 17:03:04 +02:00
Alexey Milovidov
ddff5e9145
Merge pull request #40712 from ClickHouse/ci-cmake-self-extractable
cmake: Disable default ENABLE_CLICKHOUSE_SELF_EXTRACTING
2022-08-28 17:12:19 +03:00
Alexey Milovidov
fc29303ff3
Merge pull request #40685 from ClickHouse/nlp-functional-test
Rewrite NLP tests from integration to functional
2022-08-28 17:05:42 +03:00
Alexey Milovidov
f518844f06
Merge pull request #40709 from ClickHouse/all_new_table_functions_must_be_documented
All new table functions must be documented
2022-08-28 17:05:20 +03:00
Alexey Milovidov
00074a5e14
Merge pull request #40711 from ClickHouse/all_new_functions_must_be_documented
All new functions must be documented
2022-08-28 17:05:05 +03:00
kssenii
3be79fc145 Fix 2022-08-28 15:57:46 +02:00
alesapin
0b827207db Enable zero-copy replication in CI 2022-08-28 15:46:11 +02:00
Kseniia Sumarokova
d6efbf039e
Update 02313_filesystem_cache_seeks.queries 2022-08-28 14:49:53 +02:00
Kseniia Sumarokova
a0bc5b6ea4
Merge pull request #40705 from azat/stress/max_read_buffer_size
Fix LOGICAL_ERROR with max_read_buffer_size=0 during reading marks
2022-08-28 12:41:16 +02:00
Robert Schulze
df934d8762
Merge pull request #40217 from zvonand/zvonand-minmax
Fix conversion Date32 / DateTime64 / Date to narrow types
2022-08-28 09:42:39 +02:00
Alexey Milovidov
18c56b722f Fix integration test 2022-08-28 04:30:27 +02:00
Alexey Milovidov
d3a7945b1b
Merge pull request #40646 from ClickHouse/force-documentation-3
Every profile event must have documentation
2022-08-28 04:10:11 +03:00
Alexey Milovidov
632867dbff Fix error 2022-08-28 02:40:43 +02:00
Alexey Milovidov
0f59e0511b
Merge pull request #40703 from ClickHouse/fix_02242_system_filesystem
Make 02242_system_filesystem_cache_log_table output stable
2022-08-28 03:25:44 +03:00
Alexey Milovidov
ccfce4f51a Update test 2022-08-28 02:10:04 +02:00
Alexey Milovidov
5b57c91ad6 Rename table function MeiliSearch to meilisearch 2022-08-28 02:08:51 +02:00
Alexey Milovidov
1a7dbd2c62 Update test 2022-08-28 02:05:58 +02:00
Alexey Milovidov
7831200fc8 All new table functions must be documented 2022-08-28 02:05:58 +02:00
Alexey Milovidov
b0e5174e78 Update test 2022-08-28 02:01:17 +02:00
Alexey Milovidov
0309a2ca76 Fix test 2022-08-28 01:37:23 +02:00