thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-14 10:22:10 +00:00
Code Issues Packages Projects Releases Wiki Activity
111,248 Commits 9,328 Branches 1,783 Tags 2.1 GiB
1605afe647
Commit Graph

6046 Commits

Author SHA1 Message Date
Konstantin Bogdanov
a094af9a49
Lint 2022-12-20 04:00:45 +01:00
Konstantin Bogdanov
b07001e7d7
Add a test 2022-12-20 03:26:31 +01:00
kssenii
a0c8d34775 RabbitMQ: respect flushing data by flush interval and by block size 2022-12-19 21:07:44 +01:00
Roman Vasin
8a692432de Remove unneeded headers from test.py 2022-12-19 11:48:10 +00:00
Roman Vasin
c26c257c57 Refactor test.py 2022-12-19 11:39:30 +00:00
Roman Vasin
64c0c6ae99 Refactor test_kerberos_auth test 2022-12-19 11:28:49 +00:00
Antonio Andelic
55dd591392 Try restarting ZK cluster on failed connection 2022-12-19 09:28:59 +00:00
Antonio Andelic
0162632275
Merge pull request #44306 from ClickHouse/another-fix-for-test-server-reload 
Another fix `test_server_reload`
 2022-12-19 09:14:03 +01:00
Nikolay Degterinsky
7a29bf0e4c
Merge pull request #43993 from evillique/fix_settings_constraints 
Fix CREATE query constraints
 2022-12-16 18:01:04 +01:00
Anton Popov
8b9b8b083c
Merge pull request #43726 from CurtizJ/optimize-storage-s3 
Improve performance of storage `S3` with large number of small files
 2022-12-16 14:38:10 +01:00
Antonio Andelic
372c187773 Correctly compare get result 2022-12-16 09:48:47 +00:00
Roman Vasin
775c700c24 Add second test for the case when there is no keytab 2022-12-16 08:20:01 +00:00
Nikolay Degterinsky
1861e670e9
Merge branch 'master' into fix_settings_constraints 2022-12-15 21:54:02 +01:00
Kruglov Pavel
c5b2e4cc23
Merge branch 'master' into improve-streaming-engines 2022-12-15 18:44:35 +01:00
Roman Vasin
5491aa9627 Implement wait_kerberos_kdc_is_available 2022-12-15 11:50:23 +00:00
Anton Popov
c4bcfe54cd fix test 2022-12-15 04:19:16 +00:00
Vitaly Baranov
fb8aca8319
Merge pull request #44158 from vitlibar/improve-referential-deps 
Improve referential dependencies
 2022-12-14 21:17:02 +01:00
Roman Vasin
e2ced517dd Add integration test test_kerberos_auth 2022-12-14 13:39:23 +00:00
kssenii
ea5b06023a Merge remote-tracking branch 'refs/remotes/upstream/master' into allow-single-disk-instead-of-storage-policy 2022-12-14 13:59:51 +01:00
Sema Checherinda
fdecb85dc4
Merge pull request #44156 from CheSema/fix_flaky_test_attach_detach 
make calls to be sure that parts are deleted
 2022-12-14 12:39:39 +01:00
Antonio Andelic
b2cce09004
Merge pull request #44184 from ClickHouse/close-grpc-channel-in-tests 
Close GRPC channels in tests
 2022-12-14 09:47:36 +01:00
Anton Popov
3008e701ea fix tests 2022-12-14 01:36:03 +00:00
Anton Popov
cce3257f39
Merge branch 'master' into optimize-storage-s3 2022-12-13 21:35:12 +01:00
Nikolay Degterinsky
809f15ac81
Merge branch 'master' into fix_settings_constraints 2022-12-13 20:33:16 +01:00
Nikolay Degterinsky
443bc3e656
Merge pull request #43719 from evillique/password_complexity_rules 
Add password complexity rules
 2022-12-13 17:56:45 +01:00
Anton Popov
0c87031e80 Merge remote-tracking branch 'upstream/master' into HEAD 2022-12-13 16:33:21 +00:00
Mikhail f. Shiryaev
9879d03bc4
Use contextmanager for grpc_channels 2022-12-13 12:43:48 +01:00
Antonio Andelic
b8f76ce0e5 Close GRPC channels in tests 2022-12-13 10:25:54 +00:00
Vitaly Baranov
5c1f490b3a Implement referential dependencies for table engine "Distributed" 
and for functions cluster() and clusterAllReplicas().
 2022-12-13 11:09:42 +01:00
Nikolay Degterinsky
f352403af2 Fix style 2022-12-13 09:41:31 +00:00
Nikolay Degterinsky
034269087f Fix tests 2022-12-13 09:16:23 +00:00
Anton Popov
0b396bab14 Merge remote-tracking branch 'upstream/master' into HEAD 2022-12-12 18:08:27 +00:00
Vitaly Baranov
5aaff60650 Fix referential dependencies when host & post in a clickHouse dictionary source are set by default. 2022-12-12 18:22:14 +01:00
Vitaly Baranov
76ba8ab3d4 Add new tests. 2022-12-12 18:22:09 +01:00
Anton Popov
a2fee110b8 add query for waiting of loading parts and fix test_transactions 2022-12-12 14:33:42 +00:00
Sema Checherinda
aecc509f80 make calls to be sure that parts are deleted 2022-12-12 12:37:52 +01:00
Nikolay Degterinsky
19e95e8cd8 Better implementation of password rules 2022-12-11 23:59:04 +00:00
Sema Checherinda
9a1152339c
Merge branch 'master' into do-not-run-clear-old-parts-at-shutdown 2022-12-10 09:44:56 +01:00
Nikolay Degterinsky
a8545a29a4 Better test 2022-12-10 01:32:15 +00:00
Sergei Trifonov
7c37e48092
Merge pull request #43903 from ClickHouse/merge_tree_settings_constraints 
Constraints on merge tree settings
 2022-12-09 14:16:14 +01:00
Kseniia Sumarokova
e21d4a93b8
Merge pull request #44050 from ClickHouse/fix_clickhouse_disk 
Slightly better clickhouse disks and remove DiskMemory
 2022-12-09 13:09:59 +01:00
Alexey Milovidov
68e506d48f Unify __init__.py files 2022-12-08 23:04:56 +01:00
Alexey Milovidov
f880644d3a
Merge branch 'master' into nv/fix-mutate-fetch 2022-12-09 00:51:54 +03:00
Sergei Trifonov
bae1519970
Merge branch 'master' into merge_tree_settings_constraints 2022-12-08 22:25:16 +01:00
robot-clickhouse
98ce0dbb12 Automatic style fix 2022-12-08 17:27:23 +00:00
alesapin
7a223f4956 Slightly better clickhouse disks and remove DiskMemory 2022-12-08 18:20:54 +01:00
serxa
6428b8a1bb check one more case in test 2022-12-07 20:36:04 +00:00
Alexey Milovidov
9c81fd7afc
Merge pull request #43517 from ClickHouse/password-reset-message 
Move password reset message from client to server
 2022-12-07 23:26:58 +03:00
serxa
05788245dd add test 2022-12-07 20:16:10 +00:00
Nikolay Degterinsky
8a53f5364b
Fix style 2022-12-07 10:36:16 +01:00
Nikolay Degterinsky
bb9fdc0839 Fix CREATE query constraints 2022-12-07 00:53:27 +00:00
Sema Checherinda
0b2a3195c8 adjust integration/test_partition/test_detached_part_dir_exists 2022-12-05 17:39:27 +01:00
Sema Checherinda
1f96ef57d2 remove integration/test_parts_removal 2022-12-05 17:24:54 +01:00
kssenii
d278a0dbfb Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2022-12-05 12:59:44 +01:00
kssenii
dca3444283 Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands 2022-12-04 15:56:02 +01:00
Vitaly Baranov
e1f7f04752
Referential dependencies for RESTORE (#43834) 
* Rename DDLDependencyVisitor -> DDLLoadingDependencyVisitor.

* Move building a loading graph to TablesLoader.

* Implement referential dependencies for tables and use them
when restoring tables from a backup.

* Remove StorageID::operator < (because of its inconsistency with ==).

* Add new tests.

* Fix test.

* Fix memory leak.

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
 2022-12-02 15:05:46 +01:00
Vladimir C
7d6950d397
Revert "Temporary files evict fs cache" 2022-12-02 14:50:56 +01:00
Alexander Tokmakov
edbd460997
Merge pull request #43848 from elenatorro/fix_reset_setting_on_cluster 
Add RESET_SETTING to check if it is a settings alter
 2022-12-02 15:06:28 +03:00
Alexander Tokmakov
c2a19fc354
Update test.py 2022-12-01 19:41:29 +03:00
Nikita Mikhaylov
9ef54c6ecc
Try to use exchanges during database replicated replica recovery process (#43628) 2022-12-01 13:50:56 +01:00
vdimir
08b8d335f0
add integration/test_temporary_data_in_cache 2022-12-01 11:49:25 +00:00
vdimir
7a814d2854
wip: temporary files evict fs cache 2022-12-01 11:49:24 +00:00
Elena Torró
5630412944
Merge branch 'master' into fix_reset_setting_on_cluster 2022-11-30 21:47:43 +01:00
Elena
0a35a072d9 Add RESET_SETTING to check if is a settings alter 2022-11-30 21:44:19 +01:00
Robert Schulze
6f564c59bd
Merge pull request #42641 from Enmk/fix_profile_names_mismatch 
Fixed exception when user tries to log in
 2022-11-30 13:01:57 +01:00
Anton Popov
df6d2f2163 Merge remote-tracking branch 'upstream/master' into HEAD 2022-11-29 23:21:23 +00:00
Alexey Milovidov
72bf1115e9
Merge branch 'master' into password-reset-message 2022-11-29 23:51:46 +01:00
Vasily Nemkov
3d0c07ac5b Updated tests to match improved behaviour of currentProfiles() 2022-11-30 02:32:27 +04:00
Anton Popov
2a1fd48e91 fix tests 2022-11-29 17:33:35 +00:00
robot-clickhouse
ad66e01ace Automatic style fix 2022-11-29 14:48:05 +00:00
Alexander Tokmakov
12a7d64ef5
Update test.py 2022-11-29 17:33:02 +03:00
Antonio Andelic
db46842c8a Merge branch 'master' into more-aggresive-batching-keeper 2022-11-29 12:58:33 +00:00
alesapin
a800a80ab6
Merge pull request #43770 from ClickHouse/fix_flaky_store_cleanup2 
Fix flaky `test_store_cleanup`
 2022-11-29 13:11:57 +01:00
Antonio Andelic
e39f97a255 Properly close connections and add retries 2022-11-29 09:13:23 +00:00
Anton Popov
486da48ae7 fix tests 2022-11-28 21:15:41 +00:00
Alexander Tokmakov
40d28e3489 fix flaky test 2022-11-28 18:31:22 +01:00
Alexander Tokmakov
06e6844ec0
Update test.py 2022-11-28 18:22:03 +03:00
Antonio Andelic
af7f0257c3
Merge pull request #43658 from ClickHouse/try-fix-flaky-keeper-map 
Fix flaky `KeeperMap` integration tests
 2022-11-28 12:04:04 +01:00
Nikita Taranov
8ed5cfc265
Memory bound merging for distributed aggregation in order (#40879) 
* impl

* fix style

* make executeQueryWithParallelReplicas similar to executeQuery

* impl for parallel replicas

* cleaner code for remote sorting properties

* update test

* fix

* handle when nodes of old versions participate

* small fixes

* temporary enable for testing

* fix after merge

* Revert "temporary enable for testing"

This reverts commit cce7f8884c.

* review fixes

* add bc test

* Update src/Core/Settings.h
 2022-11-28 00:41:31 +01:00
Antonio Andelic
cade21b76e Fix 4LW test 2022-11-26 19:24:10 +00:00
Antonio Andelic
65dba31c3a
Merge branch 'master' into try-fix-flaky-keeper-map 2022-11-26 19:00:30 +01:00
Sema Checherinda
15a6ce254b
Merge pull request #41145 from CheSema/lock-free-drop-partition 
Delete parts by replacing them with empty parts
 2022-11-25 12:54:30 +01:00
Antonio Andelic
f3b01d455b Use dummy pool 2022-11-25 11:12:01 +00:00
Anton Popov
c65be55da1 Merge remote-tracking branch 'upstream/master' into HEAD 2022-11-24 14:30:23 +00:00
Alexander Tokmakov
52d33fff00
Merge pull request #43366 from zzsmdfj/issue_15357_MaterializeMySQL_support_drop_mulit_table 
MaterializeMySQL_support_drop_mulit_table
 2022-11-24 14:54:24 +03:00
Sema Checherinda
6fd7dcf26f
Merge branch 'master' into lock-free-drop-partition 2022-11-24 12:34:19 +01:00
alesapin
f64f0461fe
Merge pull request #43288 from ClickHouse/Enable_compress_marks_and_primary_key 
Enable compress marks and primary key for CI
 2022-11-24 12:24:04 +01:00
Sema Checherinda
4f592bc208 fix isolation breach, droping outdated part should't be ignored 2022-11-23 15:16:09 +00:00
Sema Checherinda
f0294cd6ed fix test_merge_tree_s3_restore 2022-11-23 15:16:09 +00:00
Sema Checherinda
597421014e fix style check 2022-11-23 15:16:08 +00:00
Sema Checherinda
db86cd0f0a fix several critical races, fix tests 2022-11-23 15:16:08 +00:00
Sema Checherinda
49539e6862 fix test_partition 2022-11-23 15:16:08 +00:00
Sema Checherinda
62d3e9da4d fix test_merge_tree_hdfs 2022-11-23 15:16:08 +00:00
Sema Checherinda
d43d7ed6ea fix test_merge_tree_empty_parts and test_ttl_replicated 2022-11-23 15:16:08 +00:00
Sema Checherinda
dd93d331c1 fix 01168_mutations_isolation 2022-11-23 15:16:08 +00:00
Sema Checherinda
9edad4a078 make test_detached_parts_metrics wait delete inactive and empty parts 2022-11-23 15:16:08 +00:00
Sema Checherinda
44a8f2f460 work with test_merge_tree_s3 and test_merge_tree_s3_restore 2022-11-23 15:16:08 +00:00
Sema Checherinda
cddf44034d address test_transactions 2022-11-23 15:16:08 +00:00
Sema Checherinda
dc19141d84 fix test_detached_parts_metrics 2022-11-23 15:16:08 +00:00
Sema Checherinda
15012c7070 address the review comments 2022-11-23 15:16:08 +00:00
Sema Checherinda
fb86019db9 Delete parts over replacing them with empty parts 2022-11-23 15:16:08 +00:00
Alexander Tokmakov
f44eb4e021
Update test.py 2022-11-23 14:10:49 +03:00
kssenii
6044a9257c Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands 2022-11-23 12:00:55 +01:00
Alexey Milovidov
bedfe68503
Merge branch 'master' into nv/fix-mutate-fetch 2022-11-22 20:06:56 +01:00
kssenii
5f649baffe Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2022-11-22 14:19:51 +01:00
Dmitry Novik
e67bda65a0
Merge pull request #43453 from ClickHouse/fix-hard-limit-in-test 
Do not reuse jemalloc memory in test_global_overcommit
 2022-11-22 13:24:34 +01:00
Alexander Tokmakov
cab33c02af
Merge branch 'master' into revert_SingleValueDataString 2022-11-22 14:27:58 +03:00
Antonio Andelic
843401c3a3
Merge pull request #43323 from ClickHouse/fix-describe-delta-lake-hudi 
Fix `DESCRIBE` for `deltaLake` and `hudi` table functions
 2022-11-22 09:09:39 +01:00
kssenii
a279bac767 Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands 2022-11-21 21:49:34 +01:00
Kruglov Pavel
581e57be9f
Merge pull request #43236 from xiedeyantu/fix_hdfscluster_connection_fail 
fix skip_unavailable_shards does not work using hdfsCluster table function
 2022-11-21 19:22:16 +01:00
Igor Nikonov
47ce029cbf
Merge pull request #43326 from ClickHouse/igor/test_read_only_fast_fix 
Fix: make test_read_only_table more stable
 2022-11-21 18:25:14 +01:00
Dmitry Novik
b17dc24a94 Do not reuse jemalloc memory in test 2022-11-21 16:10:47 +00:00
Raúl Marín
35e9e169a1
Merge branch 'master' into revert_SingleValueDataString 2022-11-21 15:11:13 +01:00
Kseniia Sumarokova
a59dac6337
Merge pull request #43143 from azat/disks/s3-check-fix 
Add server UUID for disks access checks (read/read-by-offset/write/delete) to avoid possible races
 2022-11-21 15:03:46 +01:00
alesapin
b78b602ea1
Merge branch 'master' into Enable_compress_marks_and_primary_key 2022-11-21 12:05:04 +01:00
Antonio Andelic
3cb202a63b
Merge pull request #43026 from JackyWoo/keeper_manually_assign_leader 
Keeper support manually assigning leader
 2022-11-21 09:30:00 +01:00
Antonio Andelic
9a8d82f0fc Merge branch 'master' into fix-describe-delta-lake-hudi 2022-11-21 08:23:33 +00:00
zzsmdfj
a0f391745d to 15357_MaterializeMySQL_support_drop_mulit_table-fix style-black 2022-11-21 11:48:57 +08:00
Azat Khuzhin
3e42ffd372 tests: fix hdfs disks (chicken and an egg problem with current cluster.py) 
Right now cluster.py first creates HDFS and then creates clickhouse in
one API call, so you cannot interract and add missing mkdirs for the
clickhouse, fix this by using root dir where it is possible.
 2022-11-20 16:28:35 +01:00
Azat Khuzhin
0678fba3d1 tests: fix test_disk_types (by adding mkdir for HDFS) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-20 16:28:35 +01:00
Kseniia Sumarokova
f0dbfbb0f4
Merge pull request #42800 from azat/disks/web-fix 
Do not suppress exceptions in web disk (and fix retries for requests from web disk)
 2022-11-20 16:07:45 +01:00
Alexander Tokmakov
9464c32bb8 Merge branch 'master' into revert_SingleValueDataString 2022-11-19 18:00:04 +01:00
kssenii
08e4805c99 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2022-11-19 13:26:56 +01:00
Azat Khuzhin
c029549859 Allow to drop tables from s3_plain disk (so as from web disk) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-19 10:10:27 +01:00
zzsmdfj
e177927efb to 15357_MaterializeMySQL_support_drop_mulit_table-fix code-style 2022-11-19 12:05:27 +08:00
alesapin
c7d648174b
Merge pull request #43328 from ClickHouse/better-format-version-check 
Check content of `format_version` file in `MergeTreeData`
 2022-11-18 22:34:37 +01:00
Vitaly Baranov
a348332eab
Merge pull request #43227 from vitlibar/improve-masking-sensitive-info 
Improve masking sensitive info
 2022-11-18 15:37:50 +01:00
zzsmdfj
9239f50a87
Merge branch 'master' into issue_15357_MaterializeMySQL_support_drop_mulit_table 2022-11-18 21:33:31 +08:00
zzsmdfj
6a998917b0 to MaterializeMySQL_support_drop_mulit_table 2022-11-18 21:29:13 +08:00
alesapin
06894534e8 Followup fix 2022-11-17 18:52:18 +01:00
robot-clickhouse
8c55679b01 Automatic style fix 2022-11-17 17:13:23 +00:00
alesapin
cecf032b9e Disable compressed marks for some tests 2022-11-17 18:06:59 +01:00
kssenii
5cdf216c8f Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2022-11-17 16:39:36 +01:00
kssenii
69d71a8e5f Mask collection values in logs 2022-11-17 16:01:42 +01:00
robot-clickhouse
926b35b7c4 Automatic style fix 2022-11-17 14:46:25 +00:00
kssenii
5d9b34b59d More tests 2022-11-17 15:34:35 +01:00
robot-clickhouse
13e051a5de Automatic style fix 2022-11-17 13:46:21 +00:00
Antonio Andelic
d41af8b741 Check and compare content of format_version file 2022-11-17 13:41:36 +00:00
Igor Nikonov
d2d6e75d97 Fix: make test_read_only_table more stable 
+ add retries to INSERT queries after keeper node restart
 2022-11-17 13:26:32 +00:00
Nikita Taranov
7beb58b0cf
Optimize merge of uniqExact without_key (#43072) 
* impl for uniqExact

* rm unused (read|write)Text methods

* fix style

* small fixes

* impl for variadic uniqExact

* refactor

* fix style

* more agressive inlining

* disable if max_threads=1

* small improvements

* review fixes

* Revert "rm unused (read|write)Text methods"

This reverts commit a7e7480584.

* encapsulate is_able_to_parallelize_merge in Data

* encapsulate is_exact & argument_is_tuple in Data
 2022-11-17 13:19:02 +01:00
robot-clickhouse
1ad7362db7 Automatic style fix 2022-11-17 11:54:13 +00:00
Antonio Andelic
d5848d53cc Fix DESCRIBE for deltaLake and hudi table functions 2022-11-17 11:46:17 +00:00
kssenii
1964291fba Add tests 2022-11-17 12:35:43 +01:00
JackyWoo
5a09de279c fix test 2022-11-17 10:35:01 +08:00
chen
cb83337586
Update test.py 2022-11-17 06:41:43 +08:00
xiedeyantu
fc1d06f096 fix 2022-11-16 23:43:23 +08:00
Raúl Marín
45a611bc1d Fix style with black 2022-11-16 13:28:04 +01:00
Raúl Marín
4e7ccd1c06 Merge remote-tracking branch 'blessed/master' into revert_SingleValueDataString 2022-11-16 12:59:47 +01:00
xiedeyantu
7dc941cacd fix 2022-11-16 19:47:57 +08:00
Alexander Tokmakov
7e6fe94a8e
Merge pull request #43266 from ClickHouse/global-overcommit-test 
Make test_global_overcommit_tracker non-parallel
 2022-11-16 14:28:56 +03:00
alesapin
32358d2180
Merge pull request #43256 from ClickHouse/vdimir/msg_in_wait_zookeeper_to_start 
Better message in wait_zookeeper_to_start
 2022-11-16 11:58:53 +01:00
JackyWoo
01b63fd98a fix style 2022-11-16 12:51:51 +08:00
Anton Popov
f94182384a Merge remote-tracking branch 'upstream/master' into HEAD 2022-11-16 03:30:29 +00:00
xiedeyantu
c23cd091a3 fix 2022-11-16 09:59:44 +08:00
JackyWoo
f8d71f60a2
Merge branch 'master' into keeper_manually_assign_leader 2022-11-16 09:34:40 +08:00
xiedeyantu
8144516e53 fix 2022-11-16 09:33:46 +08:00
JackyWoo
ace40a2538 fix tests 2022-11-16 09:33:46 +08:00
xiedeyantu
3d218795be fix 2022-11-16 09:18:45 +08:00
Vitaly Baranov
2e242fa991 Add tests. 2022-11-16 01:35:11 +01:00
Dmitry Novik
fcdc9dfaac Make test_global_overcommit_tracker non-parallel 2022-11-15 20:52:18 +00:00
Vitaly Baranov
22abe9a4fb
Merge pull request #42950 from azat/disks/s3_plain/wide-part-fix 
Fixes for s3_plain disk that will allow to attach Wide parts
 2022-11-15 17:56:16 +01:00
robot-clickhouse
aaeeeaf1fc Automatic style fix 2022-11-15 16:40:01 +00:00
vdimir
03968eb694
Better message in wait_zookeeper_to_start 2022-11-15 16:17:12 +00:00
Antonio Andelic
6beab87610
Merge pull request #43080 from ucasfl/hudi-udtf 
Add table function hudi and deltaLake
 2022-11-15 15:42:25 +01:00
Kseniia Sumarokova
9fc96ab7fd
Merge branch 'master' into disks/web-fix 2022-11-15 12:16:32 +01:00
xiedeyantu
09c749fbd0 update 2022-11-15 13:30:29 +08:00
xiedeyantu
ec6698395e fix skip_unavailable_shards does not work using hdfsCluster table function 2022-11-15 13:25:15 +08:00
Kruglov Pavel
386b69acae
Merge pull request #43131 from xiedeyantu/fix_s3cluster_connection_fail 
fix skip_unavailable_shards does not work using s3Cluster table function
 2022-11-14 20:48:00 +01:00
Raúl Marín
790cfe2b7e Allow backward incompatible change for toDate32 since it was declared as such in 22.8 2022-11-14 18:10:36 +01:00
Kruglov Pavel
eacd7e1876
Merge pull request #42931 from zzsmdfj/MaterializedMySQL_improvement_bug_fix 
Fix issue 42856 and MaterializedMySQL improvement
 2022-11-14 16:54:12 +01:00
flynn
5bd8e2338b
Merge branch 'master' into hudi-udtf 2022-11-14 22:28:04 +08:00
Raúl Marín
58da0398a0 Merge remote-tracking branch 'blessed/master' into revert_SingleValueDataString 2022-11-14 12:46:00 +01:00
Antonio Andelic
41c0100dd7
Merge pull request #43115 from ClickHouse/fix-4lw-command-test 
Fix `test_keeper_four_word_command::test_cmd_stat`
 2022-11-14 08:33:11 +01:00
Azat Khuzhin
f2260ab724 Add a test for unavalable web server and web disk 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-12 13:25:10 +01:00
Azat Khuzhin
de5d777ca2 tests/integration: add ability not to retry server start 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-12 13:20:51 +01:00
Igor Nikonov
ee5756b5b4
Merge pull request #43122 from ClickHouse/revert-43116-revert-42607-igor/insert_zk_retries_retry 
Revert "Revert " Keeper retries during insert (clean)""
 2022-11-11 18:22:38 +01:00
Raúl Marín
5a3f324311 Try adding compatibility on write instead of on read 2022-11-11 18:12:34 +01:00
Dmitry Novik
31794284cb
Merge pull request #43105 from ClickHouse/overcommit-test 
Rewrite tests for memory overcommit
 2022-11-11 17:34:53 +01:00
xiedeyantu
5504f3af9b fix skip_unavailable_shards does not work using s3Cluster table function 2022-11-12 00:03:36 +08:00
Anton Kozlov
ccede70b5c
Merge branch 'master' into tonickkozlov/add-interserver-dns-retries 2022-11-11 14:54:30 +00:00
Anton Kozlov
024d9382c7 CLICKHOUSE-2375 Add interserver DNS retries 2022-11-11 14:47:26 +00:00
Kruglov Pavel
4ecbcbf5e1
Merge branch 'master' into MaterializedMySQL_improvement_bug_fix 2022-11-11 12:54:02 +01:00
zzsmdfj
a9a208c3cd to MaterializedMySQL_improvement_bug_fix-fix code style 2022-11-11 11:46:22 +08:00
zzsmdfj
227dcd68ba Merge branch 'MaterializedMySQL_improvement_bug_fix' of github.com:zzsmdfj/ClickHouse into MaterializedMySQL_improvement_bug_fix 2022-11-11 11:25:03 +08:00
zzsmdfj
0e614aec12 to MaterializedMySQL_improvement_bug_fix-fix code sytle 2022-11-11 11:24:29 +08:00
zzsmdfj
b01eea5c10
Merge branch 'master' into MaterializedMySQL_improvement_bug_fix 2022-11-11 10:45:36 +08:00
zzsmdfj
d4e270b522 to MaterializedMySQL_improvement_bug_fix-fix codestyle 2022-11-11 10:44:16 +08:00
Stig Bakken
9020631826 Ignore SAVEPOINT queries in MaterializedMySQL 2022-11-10 23:27:21 +01:00
Azat Khuzhin
afc6c8bb73 tests: add ATTACH FROM s3_plain for wide part 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-10 17:42:32 +01:00
robot-clickhouse
fe283af82b Automatic style fix 2022-11-10 16:08:05 +00:00
Dmitry Novik
4e15d648e7 Add test_global_overcommit_tracker/__init__.py 2022-11-10 16:02:45 +00:00
Dmitry Novik
b1e942e14c Do not run global test with sanitizers 2022-11-10 16:00:10 +00:00
Igor Nikonov
f8c907c463
Revert "Revert " Keeper retries during insert (clean)"" 2022-11-10 13:14:04 +01:00
Alexander Tokmakov
fb8c527a2f
Revert " Keeper retries during insert (clean)" 2022-11-10 14:12:24 +03:00
Antonio Andelic
c50e795788 Fix expected ZXID 2022-11-10 11:04:51 +00:00
Yakov Olkhovskiy
c08826d174
Merge branch 'master' into tests-connection-error 2022-11-10 02:57:10 -05:00
JackyWoo
8e881e6009
Merge branch 'master' into keeper_manually_assign_leader 2022-11-10 14:38:04 +08:00
flynn
da77199bf5 fix style 
fix style
 2022-11-10 03:35:38 +00:00
flynn
6cbfab4e95 fix test 2022-11-10 03:10:25 +00:00
robot-clickhouse
eb612a699d Automatic style fix 2022-11-10 01:08:12 +00:00
Dmitry Novik
981ece08ed Rewrite tests for memory overcommit 2022-11-10 00:55:39 +00:00
flynn
0ffdf3e0c7 fix style 2022-11-09 12:42:13 +00:00
flynn
c226a3cc9a fix style 2022-11-09 12:32:12 +00:00
JackyWoo
6a2957c269 fix test 2022-11-09 19:15:56 +08:00
flynn
592f6b31d9 fix and add test 2022-11-09 10:04:53 +00:00
Igor Nikonov
345304abe0 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-08 20:26:50 +00:00
kssenii
50c4beeef6 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2022-11-08 14:45:42 +01:00
Mikhail f. Shiryaev
cca2620c44
Merge pull request #42891 from ClickHouse/sync-integration-logs 
Wait for all files are in sync before archiving them
 2022-11-08 13:30:32 +01:00
JackyWoo
8f57ecf595 fix style 2022-11-08 19:56:20 +08:00
JackyWoo
a9d59a7e3f Simplify keeper_utils.is_leader method 2022-11-08 19:43:36 +08:00
Kseniia Sumarokova
d2b777c059
Merge pull request #41054 from rubin-do/hudi 
Add Hudi and DeltaLake table engines
 2022-11-08 12:03:06 +01:00
JackyWoo
9060141f16 revert last change 2022-11-08 18:44:43 +08:00
JackyWoo
eeedd7d9ba Keeper support manually assigning leader 2022-11-08 12:01:53 +08:00
Kseniia Sumarokova
4fc7926c16
Merge branch 'master' into system_table_add_column 2022-11-07 23:36:57 +01:00
alesapin
e6c3ea7ab8
Merge pull request #42890 from AVMusorin/s3_plain_system_name 
changed type name for s3 plain storage
 2022-11-07 22:46:25 +01:00
alesapin
257e766e0e Trying to fix two flaky tests 2022-11-07 20:49:41 +01:00
kssenii
b5b25c2676 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2022-11-07 17:02:41 +01:00
kssenii
819be6bb32 Merge remote-tracking branch 'upstream/master' into hudi 2022-11-07 11:30:53 +01:00
Antonio Andelic
9ac829d4c4
Merge pull request #41766 from JackyWoo/manual_snapshot_for_keeper 
Manually snapshot creation for keeper
 2022-11-07 09:18:58 +01:00
Antonio Andelic
e08f94d0f8
Merge pull request #42659 from ClickHouse/optimize-entire-partition 
Support for optimizing old parts for entire partition only
 2022-11-07 09:06:11 +01:00
Yakov Olkhovskiy
11bc712c8e add retries on ConnectionError 2022-11-07 07:12:55 +00:00
Igor Nikonov
503649a638
Merge branch 'master' into igor/insert_zk_retries_retry 2022-11-06 22:55:13 +01:00
Kseniia Sumarokova
cd5c6acbd2
Merge pull request #42960 from kssenii/fix-pg-db-datetime-table-attach 
Fix postgres db engine attaching table with datetime
 2022-11-06 22:39:15 +01:00
robot-clickhouse
f44e4fe7c6 Automatic style fix 2022-11-05 22:36:32 +00:00
Igor Nikonov
e94b9cd003 Fixes 
+ enable retries in integration test
+ typo in fault injection traces
 2022-11-05 22:28:32 +00:00
JackyWoo
9404e089fc
Merge branch 'master' into manual_snapshot_for_keeper 2022-11-05 17:37:36 +08:00
Igor Nikonov
859022660e Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-04 23:31:51 +00:00
Aleksandr Musorin
5cb69d8a22 changed type name for S3_Plain storage 
renamed a disk for S3PlainObjectStorage in system.disks table from s3 to s3_plain
 2022-11-04 17:35:51 +01:00
Vitaly Baranov
52b1f4aed9
Merge pull request #42484 from vitlibar/mask-sensitive-info-in-logs 
Mask some information in logs
 2022-11-04 14:09:38 +01:00
Kruglov Pavel
a7a5a70189
Merge pull request #42814 from kgoralski/failed_metrics_for_async_inserts 
Add `FailedAsyncInsertQuery` event metric for async inserts
 2022-11-04 13:41:39 +01:00
alesapin
82729a88c8
Merge pull request #42833 from vitlibar/backup-s3-increase-upload-part-size-exponentially 
Increase the size of upload part exponentially for backup to S3.
 2022-11-04 13:41:10 +01:00
kssenii
c9b4bc66b5 Fiux 2022-11-04 12:19:30 +01:00
Krzysztof Góralski
f526ddba68 Update tests/integration/test_failed_async_inserts/test.py 
Co-authored-by: Kruglov Pavel <48961922+Avogar@users.noreply.github.com>
 2022-11-03 22:26:24 +01:00
Krzysztof Góralski
14e922aca1 style check fixes test.py 2022-11-03 22:26:24 +01:00
Krzysztof Góralski
69f56d8d68 style check fixes 2022-11-03 22:26:24 +01:00
Krzysztof Góralski
be53a3596c Review: Failed queries metrics for Async Inserts, added integration test instead of stateless one 2022-11-03 22:26:24 +01:00
Daniil Rubin
99d57325bb Merge branch 'master' into hudi 2022-11-03 19:29:23 +00:00
Kruglov Pavel
b124875257
Merge branch 'master' into improve-streaming-engines 2022-11-03 13:22:06 +01:00
zzsmdfj
eeb9c57c59 to fix issue 42856 and MaterializedMySQL improvement 2022-11-03 20:21:02 +08:00
Antonio Andelic
66ffdda973
Merge branch 'master' into optimize-entire-partition 2022-11-03 09:52:13 +01:00
Antonio Andelic
25b19e6ba0
Merge branch 'master' into manual_snapshot_for_keeper 2022-11-03 08:55:01 +01:00
Igor Nikonov
9a315b7899 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-02 15:32:54 +00:00
Vitaly Baranov
e0133688bc
Merge branch 'master' into mask-sensitive-info-in-logs 2022-11-02 16:26:13 +01:00
avogar
9f39a6a049 Fix possible heap-use-after-free 2022-11-02 14:17:48 +00:00
Mikhail f. Shiryaev
550c776ffe
Wait for all files are in sync before archiving them 2022-11-02 13:16:27 +01:00
Vitaly Baranov
b9f2f17331 Add test and logging. 2022-11-01 12:23:20 +01:00
Vitaly Baranov
73f5664109 Fix tests. 2022-11-01 11:10:58 +01:00
Igor Nikonov
9cc64a0fcc Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-31 18:21:21 +00:00
Vitaly Baranov
d1c69a0753 Avoid changing query if it doesn't contain password. 2022-10-31 17:04:44 +01:00
Azat Khuzhin
8345c5f881 tests: cover ATTACH of BACKUP from s3_plain disk 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Vitaly Baranov
901ae126e1 Improve test test_on_cluster. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
aa5cfd6980 Improve tests. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
854e1e918f Use assert_cast() instead of as() in some places. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
43efbad642 Wipe passwords from distributed queries too. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
dcf8724adf Move prepareQueryForLogging() to a separate header. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
5d2a222fe4 Mask sensitive information in logs. 2022-10-31 10:50:33 +01:00
avogar
8e13d1f1ec Improve and refactor Kafka/StorageMQ/NATS and data formats 2022-10-28 16:41:10 +00:00
Azat Khuzhin
7a5432feaa Fix possible SIGSEGV for web disks when file does not exists 
It can be triggered in multiple ways, either when file does not exists
and you are trying to create MergeTree table from web (that has special
code for UUID handling) or by simply OPTIMIZE TABLE FINAL for MergeTree
table that is located on web disk, in both cases you will get the
following:

<details>

<summary>stacktrace</summary>

    2022.10.28 14:08:40.631226 [ 6043 ] {6165bf5f-e76b-4bca-941c-7c7ff5e3b46b} <Trace> ContextAccess (default): Access granted: OPTIMIZE ON default.data_from_web
    2022.10.28 14:08:40.632017 [ 6043 ] {6165bf5f-e76b-4bca-941c-7c7ff5e3b46b} <Debug> default.data_from_web (a3e65e1f-5fd4-47ed-9dbd-307f2586b52d) (MergerMutator): Selected 1 parts from all_1_1_0 to all_1_1_0
    2022.10.28 14:08:40.632496 [ 6043 ] {6165bf5f-e76b-4bca-941c-7c7ff5e3b46b} <Trace> default.data_from_web (a3e65e1f-5fd4-47ed-9dbd-307f2586b52d): Trying to reserve 1.00 MiB using storage policy from min volume index 0
    2022.10.28 14:08:40.632752 [ 6043 ] {6165bf5f-e76b-4bca-941c-7c7ff5e3b46b} <Trace> DiskObjectStorage(DiskWebServer): Reserved 1.00 MiB on remote disk `web_disk`, having unreserved 16.00 EiB.
    2022.10.28 14:08:40.634155 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Debug> MergeTask::PrepareStage: Merging 1 parts: from all_1_1_0 to all_1_1_0 into Compact
    2022.10.28 14:08:40.634498 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Trace> WebObjectStorage: Loading metadata for directory: http://127.0.0.1:8080/store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1
    2022.10.28 14:08:40.635025 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Trace> DiskWeb: Adding directory: store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1/
    2022.10.28 14:08:40.635355 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Trace> ReadWriteBufferFromHTTP: Sending request to http://127.0.0.1:8080/store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1/.index
    2022.10.28 14:08:40.639618 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Trace> DiskWeb: Cannot load disk metadata. Error: Code: 86. DB::Exception: Received error from remote server /store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1/.index. HTTP status code: 404 Not Found, body: <!doctype html><html><head><title>404 Not Found</title><style>
    body { background-color: #fcfcfc; color: #333333; margin: 0; padding:0; }
    h1 { font-size: 1.5em; font-weight: normal; background-color: #9999cc; min-height:2em; line-height:2em; border-bottom: 1px inset black; margin: 0; }
    h1, p { padding-left: 10px; }
    code.url { background-color: #eeeeee; font-family:monospace; padding:0 2px;}
    </style>
    </head><body><h1>Not Found</h1><p>The requested resource <code class="url">/store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1/.index</code> was not found on this server.</p></body></html>: while loading disk metadata. (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER) (version 22.11.1.1)
    2022.10.28 14:08:40.640527 [ 5488 ] {} <Trace> BaseDaemon: Received signal 11
    2022.10.28 14:08:40.641529 [ 9027 ] {} <Fatal> BaseDaemon: ########################################
    2022.10.28 14:08:40.642759 [ 9027 ] {} <Fatal> BaseDaemon: (version 22.11.1.1, build id: 12145DA78CE5E9EBB10A034177FAE5967EF81A4A) (from thread 6043) (query_id: a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1) (query: optimize table data_from_web final) Received signal Segmentation fault (11)
    2022.10.28 14:08:40.643260 [ 9027 ] {} <Fatal> BaseDaemon: Address: NULL pointer. Access: read. Unknown si_code.
    2022.10.28 14:08:40.643769 [ 9027 ] {} <Fatal> BaseDaemon: Stack trace: 0x7ffff416c0f2 0x7ffff7cd1ca8 0x7ffff679ae5e 0x7fffd52e7906 0x7fffd50c65aa 0x7fffca7a0d42 0x7fffcaee79ec 0x7fffcaf242f8 0x7fffcaf242b5 0x7fffcaf2427d 0x7fffcaf24255 0x7fffcaf2421d 0x7ffff65c3686 0x7ffff65c2295 0x7fffcaeee2a9 0x7fffcaef2c43 0x7fffcaee3c0e 0x7fffcc4a7851 0x7fffcc4a768f 0x7fffcc4abb2d 0x7fffcfdce828 0x7fffd03e3eaa 0x7fffd03dfe3b 0x7fffc8ec42d4 0x7fffc8ed51d2 0x7ffff4bdd839 0x7ffff4bde0a8 0x7ffff48ab261 0x7ffff48a769a 0x7ffff48a6335 0x7ffff409f8fd 0x7ffff4121a60
    2022.10.28 14:08:40.644411 [ 9027 ] {} <Fatal> BaseDaemon: 4. ? @ 0x7ffff416c0f2 in ?
    2022.10.28 14:08:40.676390 [ 9027 ] {} <Fatal> BaseDaemon: 5. /src/ch/clickhouse/src/Common/StringUtils/StringUtils.cpp:9: detail::startsWith(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, char const*, unsigned long) @ 0x1ca8 in /src/ch/clickhouse/.cmake/src/Common/StringUtils/libstring_utilsd.so
    2022.10.28 14:08:40.730727 [ 9027 ] {} <Fatal> BaseDaemon: 6. /src/ch/clickhouse/src/Common/StringUtils/StringUtils.h:19: startsWith(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) @ 0x59ae5e in /src/ch/clickhouse/.cmake/src/libclickhouse_common_iod.so
    2022.10.28 14:08:40.923955 [ 9027 ] {} <Fatal> BaseDaemon: 7. /src/ch/clickhouse/src/Disks/ObjectStorages/Web/MetadataStorageFromStaticFilesWebServer.cpp:58: DB::MetadataStorageFromStaticFilesWebServer::exists(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) const @ 0x6e7906 in /src/ch/clickhouse/.cmake/src/libdbmsd.so
    2022.10.28 14:08:41.291996 [ 9027 ] {} <Fatal> BaseDaemon: 8. /src/ch/clickhouse/src/Disks/ObjectStorages/DiskObjectStorage.cpp:181: DB::DiskObjectStorage::exists(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) const @ 0x4c65aa in /src/ch/clickhouse/.cmake/src/libdbmsd.so
    2022.10.28 14:08:41.704697 [ 9027 ] {} <Fatal> BaseDaemon: 9. /src/ch/clickhouse/src/Storages/MergeTree/DataPartStorageOnDisk.cpp:74: DB::DataPartStorageOnDisk::exists() const @ 0xda0d42 in /src/ch/clickhouse/.cmake/src/libclickhouse_storages_mergetreed.so
    2022.10.28 14:08:43.032459 [ 9027 ] {} <Fatal> BaseDaemon: 10. /src/ch/clickhouse/src/Storages/MergeTree/MergeTask.cpp:147: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x14e79ec in /src/ch/clickhouse/.cmake/src/libclickhouse_storages_mergetreed.so
    ...
    Segmentation fault (core dumped)

</details>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-28 15:15:49 +02:00
Antonio Andelic
5fae8b0cc9 Merge branch 'master' into optimize-entire-partition 2022-10-28 11:44:10 +00:00
Igor Nikonov
3682913172 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-27 22:01:53 +00:00
Alexander Tokmakov
86b81a2b6f
Merge pull request #42234 from arthurpassos/fix_ares_fd_process_race_condition 
Fix c-ares crash
 2022-10-27 17:17:22 +03:00
Igor Nikonov
649e8d302c Sync with retiring branch igor/insert_zk_retries 2022-10-26 17:41:44 +00:00
Igor Nikonov
8435cba352 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-26 17:35:47 +00:00
Antonio Andelic
35a9672704 Merge branch 'master' into optimize-entire-partition 2022-10-26 11:09:25 +00:00
Antonio Andelic
97aaebfa18 Address PR comments 2022-10-26 10:06:56 +00:00
Antonio Andelic
611c2e2bd7 Support for optimizing old parts for entire partition only 2022-10-25 13:40:18 +00:00
Nikolai Kochetov
5c4444237e
Merge pull request #42617 from ClickHouse/revert-revert-41268-disable-s3-parallel-write-for-part-moves-to-disk-s3 
Revert revert 41268 disable s3 parallel write for part moves to disk s3
 2022-10-25 14:29:27 +02:00
Nikita Taranov
49f6692a2e
Adapt internal data structures to 512-bit era (#42564) 
* impl

* update tests

* fix tests
 2022-10-25 13:56:28 +02:00
Raúl Marín
0016bc2b73
Improve the time to recover keeper connections (#42541) 
Co-authored-by: Nikita Mikhaylov <nikitamikhaylov@clickhouse.com>
Co-authored-by: robot-clickhouse <robot-clickhouse@clickhouse.com>
Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
 2022-10-25 13:26:29 +02:00
JackyWoo
9a36a509fe fix test 2022-10-25 17:15:49 +08:00
Alexey Milovidov
9ee7131f67
Merge pull request #42423 from ClickHouse/optimize-partitions-in-background 
Optimize partitions in background
 2022-10-24 19:41:50 +02:00
Nikolai Kochetov
beb40e4579 Split tests. 2022-10-24 14:22:06 +00:00
Vitaly Baranov
cd158b021f
Merge pull request #42562 from vitlibar/subqueries-in-row-policy-filters 
Fix using subqueries in row policy filters
 2022-10-24 13:38:15 +02:00
Igor Nikonov
54013009d3 Code migration from #41734 to align diff and git blame. 
Initial implementation was different and it changed the entire
ReplicatedMergeTreeSink::commitPart() which change history provided by git blame.
Then RetriesControl.retryLoop() was introduced later which significantly reduces
the diff since it's like while() used before.
So, check outing the current version will keep more original history in
git blame, which is useful here
 2022-10-24 08:20:11 +00:00
JackyWoo
42e391a019 fix test 2022-10-22 22:47:03 +08:00
JackyWoo
39c88c74e8 check whether last manually created snapshot is done 2022-10-22 22:31:17 +08:00
JackyWoo
bf291790b7
Merge branch 'master' into manual_snapshot_for_keeper 2022-10-22 16:57:50 +08:00
Azat Khuzhin
dbb4f8216f Merge remote-tracking branch 'upstream/master' into build/shorten-64-to-32 
Conflicts:
- src/Functions/GregorianDate.h
 2022-10-22 08:20:05 +02:00
Nikolay Degterinsky
22f87e2bf9
Merge pull request #42549 from evillique/disks-better-logs 
Better logs in clickhouse-disks
 2022-10-22 04:31:44 +02:00
Azat Khuzhin
4eeb7337ed Merge remote-tracking branch 'upstream/master' into build/shorten-64-to-32 
Conflicts:
- src/Common/intExp.h
- src/Functions/DateTimeTransforms.h
- src/Functions/FunctionsConversion.h
 2022-10-21 22:51:56 +02:00
Alexander Tokmakov
7d11054134
Merge branch 'master' into fix_ares_fd_process_race_condition 2022-10-21 18:27:22 +03:00
Vitaly Baranov
2033ea1216 Support subqueries in row policy filters. 2022-10-21 15:42:29 +02:00
Alexander Tokmakov
5ffd026261
Merge pull request #42529 from azat/tests/integration/fix-sanitizer-reports 
Fix sanitizer reports in integration tests
 2022-10-21 16:01:43 +03:00
Alexander Tokmakov
1343f62e8f
Update test.py 2022-10-21 15:59:27 +03:00
Azat Khuzhin
0365bc56aa Add backward compatiblity check for some functions 
* State of aggregate functions

This test goes though all aggregate functions that:
- has only one argument
- support string as an argument

And do a simple check by creating the aggregate state with one string.

Yes this is not covers everything (does not cover functions with
different number of arguments, types, different states in case of
multiple values - uniqCombined, but as for uniqCombined it will be
checked via uniqHLL12), but at least something.

And about type, String had been selected, since it more likely that
there will be used some hash function internally.

* Function(String)

Check every function which has only one argument, and it's type is
String.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:43 +02:00
Antonio Andelic
3091d609fa
Merge branch 'master' into optimize-partitions-in-background 2022-10-21 11:01:41 +02:00
Nikolay Degterinsky
d7d0794d2f Better logs in clickhouse-disks 2022-10-20 23:17:29 +00:00
Nikolai Kochetov
57ffcb7ed0 Revert "Merge pull request #41681 from ClickHouse/revert-41268-disable-s3-parallel-write-for-part-moves-to-disk-s3" 
This reverts commit 7956c2becf, reversing
changes made to 57be648984.
 2022-10-20 19:51:27 +02:00
Arthur Passos
50f9acb221
Merge branch 'master' into fix_ares_fd_process_race_condition 2022-10-20 13:35:58 -03:00
Arthur Passos
2fe59e7e8f lower thread and iterations number 2022-10-20 13:30:20 -03:00
Alexander Tokmakov
905a4ded42
Merge pull request #42535 from ClickHouse/tavplubix-patch-7 
Disable `test_storage_nats`, because it's permanently broken
 2022-10-20 16:58:46 +03:00
Alexander Tokmakov
5b03bbce2e
Merge pull request #41981 from ClickHouse/fixes_for_replicated_merge_tree2 
Some fixes for ReplicatedMergeTree
 2022-10-20 16:57:51 +03:00
Alexander Tokmakov
89ff3d4731
Update test.py 2022-10-20 16:54:52 +03:00
Azat Khuzhin
c399b021cf Fix sanitizer reports in integration tests 
Before they was ignored because first there was a check for a sign of
sanitizer (==================), but it was done by
clickhouse-server.log, while sanitizer write to stderr.log.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-20 15:45:40 +02:00
Antonio Andelic
9d7e44409a
Merge pull request #41342 from ClickHouse/keeper-upload-snapshot-to-s3 
Upload Keeper snapshots to S3
 2022-10-20 09:14:59 +02:00
Antonio Andelic
77eb353839
Merge pull request #42243 from ClickHouse/use-multiread-more 
Use MultiRead where possible
 2022-10-20 09:04:54 +02:00
Alexander Tokmakov
f958c155f3
Merge branch 'master' into fixes_for_replicated_merge_tree2 2022-10-19 17:25:24 +03:00
Antonio Andelic
05e6b94b54 Polishing 2022-10-19 12:59:28 +00:00
Antonio Andelic
e5408aac99 Simplify logic 2022-10-19 12:34:20 +00:00
Arthur Passos
7f0a2d32c8 fix test errors 2022-10-18 20:15:58 -03:00
Vitaly Baranov
1365105bc4 Implement backup to S3 2022-10-19 00:04:41 +02:00
Alexander Tokmakov
ea77ff9022 Merge branch 'master' into fixes_for_replicated_merge_tree2 2022-10-18 17:25:25 +02:00
Antonio Andelic
e828dce331 Disable auto optimize by default 2022-10-18 12:11:23 +00:00
Antonio Andelic
bd2522fdd0 Merge branch 'master' into optimize-partitions-in-background 2022-10-18 09:13:59 +00:00
Antonio Andelic
ee2f5977d5
Merge branch 'master' into keeper-upload-snapshot-to-s3 2022-10-18 10:02:50 +02:00
chen
163b3c9f52
fix test 
fix test
 2022-10-18 14:53:21 +08:00
xiedeyantu
10acf43db4 fix test 2022-10-18 09:20:59 +08:00
Vitaly Baranov
eb637a6f81
Merge pull request #42232 from azat/backups/s3 
Support BACKUP to S3 with as-is path/data structure
 2022-10-18 02:37:07 +02:00
Yakov Olkhovskiy
3ce1fa09d6
Merge pull request #41198 from ClickHouse/composable-protocol 
Composable protocol
 2022-10-17 11:23:12 -04:00
Antonio Andelic
7c7395575f Merge branch 'master' into keeper-upload-snapshot-to-s3 2022-10-17 11:02:36 +00:00
Antonio Andelic
878dee40b1 Use ZooKeeper in a single test 2022-10-17 07:36:49 +00:00
xiedeyantu
2a356715da fix test 2022-10-16 16:46:00 +08:00
Arthur Passos
f414edebae fix black style 2022-10-14 15:54:48 -03:00
Arthur Passos
299d848473 style fix? 2022-10-14 15:23:10 -03:00
Arthur Passos
78202f9660
Merge branch 'master' into fix_ares_fd_process_race_condition 2022-10-14 12:56:39 -03:00
Arthur Passos
6d22bb78e0 custom py script in integ tests 2022-10-14 12:39:08 -03:00
Anton Popov
95fdb0a877 remove outdated parts asynchronously 2022-10-14 14:52:57 +00:00
Azat Khuzhin
8830f0608d Support BACKUP to S3 with as-is path/data structure 
Right now backup to S3 does not make a lot of sense, since:
- it has random names, and to decoding them
- requires metadata from local disk (/var/lib/disks/DISK/BACKUP_NAME)
- or send_metadata (but it is also tricky even with it)

So this patch adds simpler interface for S3, it is only suitable for
BACKUP/RESTORE, so don't try to use it for MergeTree engine.

It is done by adding separate disk - `s3_plain` for this, that:
- does not support any extended features, like renames/hardlinks/attrs/...
  (so basically everything that MergeTree requires)
- only write/read/unlink/list files

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-14 12:59:25 +02:00
nvartolomei
ea47ab1400
assert unused value in test_replicated_merge_tree_compatibility 2022-10-12 18:12:25 +01:00
Yakov Olkhovskiy
6c8e52eb67
Merge branch 'master' into composable-protocol 2022-10-08 20:33:32 -04:00
Anton Popov
9f1104c2ad do not load inactive parts 2022-10-08 01:39:46 +00:00
robot-clickhouse
3bed015a1f Automatic style fix 2022-10-08 00:09:24 +00:00
Yakov Olkhovskiy
6bb62d4d03 add test for PROXYv1 2022-10-08 00:01:58 +00:00
Vitaly Baranov
69ebf12dab
Merge pull request #42146 from azat/backups/metadata-overflow-fix 
Fix reusing of files > 4GB from base backup
 2022-10-08 00:22:28 +02:00
Alexander Tokmakov
b8d9066004 Revert "Resurrect parallel distributed insert select with s3Cluster (#41535)" 
This reverts commit 860e34e760.
 2022-10-07 15:53:30 +02:00
Azat Khuzhin
94566abda9 Fix reusing of files > 4GB from base backup 
Previosly u64 numbers was truncated to u32 numbers during writing to the
mdatadata xml file, and further incremental backup cannot reuse them,
since the file in base backup is smaller.

P.S. There can be other places, I thought about enabling
-Wshorten-64-to-32, but there are lots of warnings right now.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-07 14:56:09 +02:00
Yakov Olkhovskiy
2229087896
Merge branch 'master' into composable-protocol 2022-10-07 02:23:26 -04:00
Yakov Olkhovskiy
997fa5e295 review suggestions 2022-10-07 01:16:22 +00:00
Nikita Mikhaylov
860e34e760
Resurrect parallel distributed insert select with s3Cluster (#41535) 2022-10-06 13:47:32 +02:00
Nikita Mikhaylov
766107df0a
Fixed the restriction on maximum size of replicated fetches (#42090) 2022-10-06 13:18:46 +02:00
Kseniia Sumarokova
316824c02a
Merge pull request #42085 from elenatorro/use_send_metadata_setting_on_storage_config 
Use send_metadata instead of send_object_metadata
 2022-10-06 12:33:25 +02:00
Kseniia Sumarokova
3cff776109
Merge pull request #41254 from filimonov/system_disks_unreserved_space 
Add unreserved_space column to system.disks
 2022-10-06 12:28:23 +02:00
Elena
3914b8afda Remove 'send_metadata' setting 2022-10-05 14:54:08 +02:00
Elena
5bbd2d1729 Use send_metadata instead of send_object_metadata 2022-10-05 13:43:59 +02:00
Kseniia Sumarokova
6775c2c0a8
Update test.py 2022-10-04 16:52:53 +02:00
Kseniia Sumarokova
0b0c7e359f
Update test.py 2022-10-04 16:04:57 +02:00
Kseniia Sumarokova
39d5dfb3f6
Fix integration test 2022-10-04 16:03:11 +02:00
Kseniia Sumarokova
1ffa3291d6
Update storage_conf.xml 2022-10-04 11:18:02 +02:00
kssenii
2ab884359e More efficient WriteBufferFromAzureBlobStorage 2022-10-03 22:15:04 +02:00
kssenii
c6d7cd5820 Fix write into azure blob storage 2022-10-03 18:54:07 +02:00
Kseniia Sumarokova
0bfbb5c38a
Merge branch 'master' into system_disks_unreserved_space 2022-10-03 14:25:47 +02:00
Alexander Tokmakov
a2c29076d6 avoid detached broken part duplicates 2022-10-03 12:27:17 +02:00
Alexander Tokmakov
7b5b7e4963
Merge pull request #41945 from ClickHouse/enable_more_s3_tests 
Enable some disabled S3 tests
 2022-10-03 13:25:50 +03:00
Alexander Tokmakov
035be1feae fix 2022-10-02 14:58:33 +02:00
robot-clickhouse
4cd2dc8d0d Automatic style fix 2022-10-02 12:57:04 +00:00
Alexander Tokmakov
4ce8f47400 fix 2022-10-02 14:50:24 +02:00
Vitaly Baranov
baf48ec400
Merge pull request #41562 from vitlibar/refactor-wiping-info-from-logs 
Refactor wiping sensitive information from logs.
 2022-10-02 12:58:58 +02:00
Alexander Tokmakov
e40917a9c5 fix 2022-10-02 00:44:46 +02:00
Kseniia Sumarokova
fe6de361c6
Merge pull request #41933 from ClickHouse/kssenii-patch-5 
Fix broken test_disks_app_func
 2022-10-02 00:17:28 +02:00
Vitaly Baranov
a5b2a49ab1 Refactor wiping sensitive information from logs. 2022-10-01 17:17:45 +02:00
Alexey Milovidov
645dd997d3
Update test.py 2022-10-01 18:16:25 +03:00
robot-clickhouse
26ad8b2f13 Automatic style fix 2022-10-01 11:55:58 +00:00
Kseniia Sumarokova
3afec69021
Update test.py 2022-10-01 13:49:50 +02:00
kssenii
7fbc77eacc Avoid incorrect warning during update of DiskSelector disks 2022-09-30 17:49:18 +02:00
kssenii
229414ed09 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2022-09-30 17:22:23 +02:00
kssenii
d538b5f3b8 Allow custom disk configuration 2022-09-30 17:20:10 +02:00
kssenii
fb3bcd2fe9 Fix use_environment_credentials 2022-09-30 12:25:39 +02:00
Alexander Tokmakov
bd305960e7
Merge pull request #41898 from ClickHouse/fix-tests-sync-replica 
Another sync replicas in test_recovery_replica
 2022-09-29 19:36:20 +03:00
alesapin
0097f15ee7
Merge pull request #41832 from ClickHouse/make_copy_instead_of_hardlink 
Use copy for some operations instead of hardlinks
 2022-09-29 17:12:08 +02:00
Alexander Tokmakov
4e422b8046
Merge pull request #41741 from ClickHouse/fix_intersecting_parts 
Fix intersecting parts
 2022-09-29 18:00:22 +03:00
robot-clickhouse
acdc3504aa Automatic style fix 2022-09-29 11:21:41 +00:00
Kseniia Sumarokova
6270f513fc
Update test.py 2022-09-29 13:03:03 +02:00
Yakov Olkhovskiy
028af59702
Update tests/integration/test_recovery_replica/test.py 
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2022-09-28 12:33:38 -04:00
alesapin
905d4585ac
Merge branch 'master' into make_copy_instead_of_hardlink 2022-09-28 16:47:45 +02:00
Yakov Olkhovskiy
10f6ba90e0
add sync_replicas 2022-09-28 10:00:59 -04:00
alesapin
4b4c63749a Correct test version 2022-09-28 14:38:11 +02:00
Daniil Rubin
95b5d2d29d
Merge branch 'master' into hudi 2022-09-28 14:24:55 +03:00
alesapin
2eef384807 Add comments and test 2022-09-28 13:09:48 +02:00
Alexey Milovidov
e5ffd74a56
Merge branch 'master' into trying_to_fix_compressed_marks 2022-09-28 02:30:10 +03:00
Alexey Milovidov
12dc33f4fd
Merge branch 'master' into nv/fix-mutate-fetch 2022-09-28 01:28:19 +03:00
kssenii
01c4299481 Support single disk instead of storage policy 2022-09-27 22:34:31 +02:00
Alexander Tokmakov
0915bfdb17 fix 2022-09-27 17:42:36 +02:00
Alexander Tokmakov
b4a10c330c fix 2022-09-27 17:34:24 +02:00
Antonio Andelic
036d1c8cbc
Merge pull request #40991 from ClickHouse/start-embedded-keeper-async 
Always start embedded Keeper in async mode
 2022-09-27 14:44:19 +02:00
robot-clickhouse
08af2a32bf Automatic style fix 2022-09-27 09:53:26 +00:00
alesapin
4d2a5d752d Add failing test 2022-09-27 11:36:20 +02:00
Antonio Andelic
655bde8e3f Merge branch 'master' into keeper-upload-snapshot-to-s3 2022-09-27 07:28:26 +00:00
JackyWoo
d1c85f68e8 manually snapshot creation for keeper 2022-09-26 18:29:15 +08:00
robot-clickhouse
7a6386c732 Automatic style fix 2022-09-25 16:13:17 +00:00
Yakov Olkhovskiy
1407f6bcda add test 2022-09-25 15:46:12 +00:00
kssenii
3725397040 Try fix azure tests 2022-09-23 15:24:10 +02:00
Alexander Tokmakov
3eebceaa38
Update test.py 2022-09-23 14:01:00 +03:00
Kseniia Sumarokova
ea43cb5648
Merge pull request #41261 from kssenii/s3-header-auth 
Support s3 authorisation headers from ast arguments
 2022-09-23 12:48:08 +02:00
Kseniia Sumarokova
a0c5da3459
Merge pull request #41652 from kssenii/fix-disk-web 
Fixes for disk web
 2022-09-23 12:43:50 +02:00
kssenii
0e8343e6f3 Revert "Better config for cache" 
This reverts commit 0ca6aadbdb.
 2022-09-22 17:57:44 +02:00
kssenii
109d2ad022 Better 2022-09-22 16:22:05 +02:00
Antonio Andelic
44a3d6babe Address PR comments 2022-09-22 13:03:27 +00:00
Kseniia Sumarokova
79304422b3
Update test.py 2022-09-22 13:11:35 +02:00
kssenii
cb442a3ad7 Fixes for disk web 2022-09-21 21:26:55 +02:00
kssenii
92c2219e82 Fix test 2022-09-21 20:51:09 +02:00
Antonio Andelic
6798b500e9 Wait on startup for Keeper 2022-09-21 15:12:16 +00:00
Antonio Andelic
aa700836b8 Merge branch 'master' into start-embedded-keeper-async 2022-09-21 13:46:54 +00:00
Kseniia Sumarokova
1e7308832c
Merge branch 'master' into system_disks_unreserved_space 2022-09-21 13:04:39 +02:00
kssenii
f917b268b3 Merge remote-tracking branch 'upstream/master' into s3-header-auth 2022-09-21 12:56:25 +02:00
nvartolomei
8d55637af8
Update test.py 2022-09-21 10:10:21 +01:00
Antonio Andelic
fc6c1faa57 Merge branch 'master' into keeper-upload-snapshot-to-s3 2022-09-21 06:57:58 +00:00
nvartolomei
842b0bb44b
Explain why hostName function is used in test mutation 2022-09-20 22:41:14 +01:00
nvartolomei
3d6545cea1
Reformat test.py to match style 2022-09-20 22:04:20 +01:00
Nicolae Vartolomei
45f1f572f0 Get rid of scary setting for testing fetching byte identical parts 2022-09-20 19:27:49 +00:00
Nicolae Vartolomei
90666a43a8 CLICKHOUSE-2153: Move new_data_part shared pointer to allow Outdated part to be deleted immediately on mutation failure 2022-09-20 18:23:19 +00:00
kssenii
b8079e4577 Merge remote-tracking branch 'upstream/master' into rename-some-cache-commands 2022-09-20 13:02:18 +02:00
alesapin
8d88b68a85
Merge pull request #41415 from ClickHouse/revert_prefixes_s3 
Revert prefixes s3 + add retries on write requests unexpected errors
 2022-09-20 12:55:04 +02:00
Mikhail Filimonov
fb3ebbb2d2
Apply style fix for tests/integration/test_disk_types/test.py 2022-09-20 12:12:01 +02:00
Vitaly Baranov
f58f8104b5
Merge pull request #41474 from vitlibar/mask-some-info-in-logs 
Mask some information in logs.
 2022-09-20 12:11:02 +02:00
Alexey Milovidov
03bbf0676f
Merge branch 'master' into revert_prefixes_s3 2022-09-20 06:14:07 +03:00
Sergei Trifonov
cf2db48c29
Merge pull request #40631 from ClickHouse/readonly-settings-allow 
Allow to modify constrained settings in readonly mode
 2022-09-20 02:18:14 +02:00
Sergei Trifonov
358c5a0139
Merge pull request #41532 from ClickHouse/s3-api-calls-events-for-disk-s3 
Detailed S3 profile events `DiskS3*` per S3 API call split for S3 ObjectStorage
 2022-09-20 02:10:50 +02:00
Vitaly Baranov
b1199878ad
Merge pull request #39977 from vitlibar/improve-recovery-of-replicated-access-storage 
Improve recovery of ReplicatedAccessStorage after errors.
 2022-09-20 00:36:30 +02:00
Sergei Trifonov
7902922ae4
Merge branch 'master' into readonly-settings-allow 2022-09-19 21:00:12 +02:00
serxa
7a1d4a404a add DiskS3* profile events per S3 API calls 2022-09-19 18:10:47 +00:00
serxa
f8aa738511 more conventional profile events names 2022-09-19 17:23:22 +00:00
Vitaly Baranov
052ecaa211
Merge branch 'master' into improve-recovery-of-replicated-access-storage 2022-09-19 15:26:21 +02:00
Vitaly Baranov
f75dd93965 Mask some information in logs. 2022-09-19 15:00:06 +02:00
Sergei Trifonov
ff2e1ff8d8
Merge branch 'master' into support_s3_api_event_metrics 2022-09-19 14:35:45 +02:00
Sergei Trifonov
26da32f53f
Update tests/integration/test_profile_events_s3/test.py 2022-09-19 14:35:23 +02:00
Alexander Tokmakov
c6efb1773e
Merge branch 'master' into fix_drop_partition_on_cluster 2022-09-19 15:18:05 +03:00
kssenii
e3cd3686af Rename 2022-09-19 14:02:51 +02:00
Sergei Trifonov
f7a1ed3141
Merge branch 'master' into readonly-settings-allow 2022-09-19 12:52:36 +02:00
Azat Khuzhin
00d4ac542d tests: fix test_hive_query flakiness 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-19 07:54:49 +02:00
alesapin
1d0dbfb112
Merge pull request #41444 from ClickHouse/one_wrong_config_assert 
One more validation for broken config in keeper
 2022-09-18 15:56:48 +02:00
Alexey Milovidov
6b752bff3a
Merge branch 'master' into revert_prefixes_s3 2022-09-18 07:10:57 +03:00
alesapin
b6d61318d0 One more validation for broken config in keeper 2022-09-18 00:04:16 +02:00
robot-clickhouse
838bc678cc Automatic style fix 2022-09-17 18:38:12 +00:00
Alexey Milovidov
c480531423 Remove trash error message from integration tests 2022-09-17 20:27:19 +02:00
Alexey Milovidov
ecfd8cb5a6 Remove trash error message from integration tests 2022-09-17 20:24:56 +02:00
Vitaly Baranov
406c76ee4e Fix test. 2022-09-16 19:02:06 +02:00
Vitaly Baranov
1206e77883 Add test checking that ReplicatedAccessStorage can continue after reloading ZooKeeper. 2022-09-16 19:02:06 +02:00
alesapin
456da30c9f Revert "Merge pull request #41249 from ClickHouse/revert-40968-s3-sharding-2" 
This reverts commit ac60300997, reversing
changes made to a9c272283a.
 2022-09-16 14:06:26 +02:00
Robert Schulze
b32b02d844
Merge pull request #40897 from ClickHouse/catboost-bridge-resurrected 
Move CatBoost evaluation into clickhouse-library-bridge
 2022-09-16 13:12:09 +02:00
alesapin
4e780cf99f
Merge pull request #41378 from ClickHouse/rename_test 
Check test flakiness
 2022-09-16 11:51:04 +02:00
Antonio Andelic
296fe01fb8 Merge branch 'master' into keeper-upload-snapshot-to-s3 2022-09-16 08:00:15 +00:00
Alexey Milovidov
da01982652
Merge pull request #41046 from azat/build/llvm-15 
Switch to llvm/clang 15
 2022-09-16 07:31:06 +03:00
Alexander Tokmakov
03c193ccca fix ON CLUSTER with Replicated database cluster 2022-09-15 21:15:57 +02:00
alesapin
1dd50e3ce8 Rename test 2022-09-15 17:07:42 +02:00
Antonio Andelic
c7775b7f16 Add tests for S3 snapshot upload 2022-09-15 13:37:17 +00:00
serxa
a960fdea89 Merge branch 'readonly-settings-allow' of github.com:ClickHouse/ClickHouse into readonly-settings-allow 2022-09-15 11:38:06 +00:00
serxa
4971c1f04d review fixes 2022-09-15 11:34:14 +00:00
kssenii
559c696230 Fix 2022-09-14 20:35:28 +02:00
mateng0915
372a0b7794 added events into s3 integration test 2022-09-14 19:19:47 +08:00
alesapin
51a302a70f
Merge branch 'master' into revert-40968-s3-sharding-2 2022-09-14 11:53:14 +02:00
Mikhail Filimonov
a136465eb4
Fix test integration/test_disk_types/test.py 2022-09-14 11:08:37 +02:00
Antonio Andelic
36f46891dd
Merge pull request #41215 from ClickHouse/keeper-close-connection-on-shutdown 
Close sessions on Keeper shutdown
 2022-09-13 19:09:34 +02:00
kssenii
420ac4eb43 s3 header auth in ast 2022-09-13 15:13:28 +02:00
Antonio Andelic
baf7255cff
Merge pull request #39976 from ClickHouse/keeper-storage 
KeeperMap storage engine
 2022-09-13 14:15:09 +02:00
Alexander Tokmakov
4d146b05a9
Merge pull request #38262 from PolyProgrammist/fix-ordinary-system-unfreeze 
Fix SYSTEM UNFREEZE for ordinary database
 2022-09-13 14:55:11 +03:00
alesapin
9af591a328
Revert "Sharding s3 key names (2)" 2022-09-13 12:29:02 +02:00
Antonio Andelic
45cde90219 Fix test 2022-09-13 09:51:31 +00:00
Sergei Trifonov
9cd78585c3 fix tests 2022-09-13 00:12:40 +02:00
Robert Schulze
fac1be9700
chore: restore SYSTEM RELOAD MODEL(S) and moniting view SYSTEM.MODELS 
- This commit restores statements "SYSTEM RELOAD MODEL(S)" which provide
  a mechanism to update a model explicitly. It also saves potentially
  unnecessary reloads of a model from disk after it's initial load.

  To keep the complexity low, the semantics of "SYSTEM RELOAD MODEL(S)
  was changed from eager to lazy. This means that both statements
  previously immedately reloaded the specified/all models, whereas now
  the statements only trigger an unload and the first call to
  catboostEvaluate() does the actual load.

- Monitoring view SYSTEM.MODELS is also restored but with some obsolete
  fields removed. The view was not documented in the past and for now it
  remains undocumented. The commit is thus not considered a breach of
  ClickHouse's public interface.
 2022-09-12 19:33:02 +00:00
Azat Khuzhin
b698a4ff65 Apply changes to http handlers on fly without server restart 
This has been implemented by simply restarting http servers in case of
http_handlers directive in configuration xml had been changed.

But, for this I have to change the handlers interface to accept
configuration separatelly, since the configuration that contains in the
server is the configuration with which server had been started.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Co-authored-by: Antonio Andelic
 2022-09-12 17:34:51 +02:00
Antonio Andelic
deee0d639f Address PR comments 2022-09-12 14:43:52 +00:00
Sergei Trifonov
fe88a7991d
Merge branch 'master' into readonly-settings-allow 2022-09-12 16:20:40 +02:00
Antonio Andelic
1cd43d8e5c Merge branch 'master' into keeper-storage 2022-09-12 13:42:51 +00:00
Antonio Andelic
e985ee3354 Close sessions on Keeper shutdown 2022-09-12 12:22:48 +00:00
kssenii
02c3d8b0a0 Fix 2022-09-11 16:42:56 +02:00
Alexey Milovidov
5802c2fdd2
Merge pull request #40713 from ClickHouse/remove-useless-line 
Remove one line from XML, because we do not care
 2022-09-11 09:53:30 +03:00
Alexey Milovidov
a5388affb8
Merge pull request #41023 from ClickHouse/remove-trash-3 
Remove trash from config
 2022-09-11 09:52:45 +03:00
Alexey Milovidov
91338ea771
Merge branch 'master' into remove-useless-line 2022-09-11 02:03:39 +03:00
Azat Khuzhin
46eafb4732 tests: skip test_send_crash_reports under ASan 
Since it also does not fits into timeouts [1]:

    2022-09-08 21:16:20 [ 377 ] DEBUG : Command:['docker', 'exec', '-u', 'root', 'roottestsendcrashreports_node_1', 'bash', '-c', 'pkill -SEGV clickhouse'] (cluster.py:95, run_and_check)
    ...
    2022-09-08 21:16:22 [ 377 ] DEBUG : run container_id:roottestsendcrashreports_node_1 detach:False nothrow:False cmd: ['cat', '/result.txt'] (cluster.py:1744, exec_in_container)
    ...
    2022-09-08 21:16:36 [ 377 ] DEBUG : Stdout:INITIAL_STATE (cluster.py:103, run_and_check)

And server logs:

    2022.09.08 21:16:21.112076 [ 228 ] {} <Fatal> BaseDaemon: ########################################
    2022.09.08 21:16:21.112170 [ 228 ] {} <Fatal> BaseDaemon: (version 22.9.1.1, build id: 0F7336E0A4D64134C51C8365DADCB78A9B39AA3B) (from thread 1) (no query) Received signal Segmentation fault (11)
    2022.09.08 21:16:21.112244 [ 228 ] {} <Fatal> BaseDaemon: Address: 0xde Access: read. Unknown si_code.
    2022.09.08 21:16:21.112321 [ 228 ] {} <Fatal> BaseDaemon: Stack trace: 0x7fbe21d09376 0x40a4f71a 0xe293a4b 0xdc5c51a 0x38dee227 0xdc326a0 0x38e319d9 0xdc2b4e2 0xdc25fdb 0x7fbe21b2c083 0xdb636ae
    2022.09.08 21:16:21.112419 [ 228 ] {} <Fatal> BaseDaemon: 3. pthread_cond_wait @ 0x7fbe21d09376 in ?
    2022.09.08 21:16:21.122914 [ 228 ] {} <Fatal> BaseDaemon: 4. ./build_docker/../contrib/libcxx/src/condition_variable.cpp:0: std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) @ 0x40a4f71a in /usr/bin/clickhouse
    2022.09.08 21:16:21.233016 [ 228 ] {} <Fatal> BaseDaemon: 5.1. inlined from ./build_docker/../contrib/libcxx/include/atomic:952: unsigned long std::__1::__cxx_atomic_load<unsigned long>(std::__1::__cxx_atomic_base_impl<unsigned long> const*, std::__1::memory_order)
    2022.09.08 21:16:21.233135 [ 228 ] {} <Fatal> BaseDaemon: 5.2. inlined from ../contrib/libcxx/include/atomic:1582: std::__1::__atomic_base<unsigned long, false>::load(std::__1::memory_order) const
    2022.09.08 21:16:21.233183 [ 228 ] {} <Fatal> BaseDaemon: 5.3. inlined from ../contrib/libcxx/include/atomic:1586: std::__1::__atomic_base<unsigned long, false>::operator unsigned long() const
    2022.09.08 21:16:21.233234 [ 228 ] {} <Fatal> BaseDaemon: 5.4. inlined from ../src/Daemon/BaseDaemon.cpp:967: operator()
    2022.09.08 21:16:21.233303 [ 228 ] {} <Fatal> BaseDaemon: 5.5. inlined from ../contrib/libcxx/include/__mutex_base:402: void std::__1::condition_variable::wait<BaseDaemon::waitForTerminationRequest()::$_0>(std::__1::unique_lock<std::__1::mutex>&, BaseDaemon::waitForTerminationRequest()::$_0)
    2022.09.08 21:16:21.233334 [ 228 ] {} <Fatal> BaseDaemon: 5. ../src/Daemon/BaseDaemon.cpp:967: BaseDaemon::waitForTerminationRequest() @ 0xe293a4b in /usr/bin/clickhouse
    2022.09.08 21:16:21.350675 [ 228 ] {} <Fatal> BaseDaemon: 6. ./build_docker/../programs/server/Server.cpp:0: DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0xdc5c51a in /usr/bin/clickhouse
    2022.09.08 21:16:21.394092 [ 228 ] {} <Fatal> BaseDaemon: 7. ./build_docker/../contrib/poco/Util/src/Application.cpp:0: Poco::Util::Application::run() @ 0x38dee227 in /usr/bin/clickhouse
    2022.09.08 21:16:21.654195 [ 228 ] {} <Fatal> BaseDaemon: 8. ./build_docker/../programs/server/Server.cpp:466: DB::Server::run() @ 0xdc326a0 in /usr/bin/clickhouse
    2022.09.08 21:16:21.666991 [ 228 ] {} <Fatal> BaseDaemon: 9. ./build_docker/../contrib/poco/Util/src/ServerApplication.cpp:0: Poco::Util::ServerApplication::run(int, char**) @ 0x38e319d9 in /usr/bin/clickhouse
    2022.09.08 21:16:21.916078 [ 228 ] {} <Fatal> BaseDaemon: 10. ./build_docker/../programs/server/Server.cpp:0: mainEntryClickHouseServer(int, char**) @ 0xdc2b4e2 in /usr/bin/clickhouse
    2022.09.08 21:16:21.929922 [ 228 ] {} <Fatal> BaseDaemon: 11. ./build_docker/../programs/main.cpp:0: main @ 0xdc25fdb in /usr/bin/clickhouse
    2022.09.08 21:16:21.929981 [ 228 ] {} <Fatal> BaseDaemon: 12. __libc_start_main @ 0x7fbe21b2c083 in ?
    2022.09.08 21:16:30.357032 [ 228 ] {} <Fatal> BaseDaemon: 13. _start @ 0xdb636ae in /usr/bin/clickhouse
    2022.09.08 21:16:31.383233 [ 228 ] {} <Fatal> BaseDaemon: Integrity check of the executable skipped because the reference checksum could not be read. (calculated checksum: 6200AC7C1270DC293DF3302E1C64399B)
    ...
    2022.09.08 21:16:40.564453 [ 228 ] {} <Information> SentryWriter: Sending crash report

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/41046/a0b85eaca8d4003c9fbc4571b30830d30f1984e9/integration_tests__asan__[3/3].html

Though another option is to increase waiting time.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-10 21:38:35 +02:00
robot-clickhouse
5bc2845e49 Automatic style fix 2022-09-10 03:17:34 +00:00
Alexey Milovidov
f2227e125f Fix test 2022-09-10 05:11:09 +02:00
Alexey Milovidov
61278c81e4 Merge branch 'master' into remove-useless-line 2022-09-10 05:07:10 +02:00
robot-clickhouse
79ccce23e9 Automatic style fix 2022-09-09 20:09:54 +00:00
Igor Nikonov
dbfb448290 Fix: integration test, standalone keeper mode 
There is logic regarding which keeper binary use to start keeper cluster in an integration test
There 2 options:
(1) standalone keeper binary (expected binary name clickhouse-keeper)
(2) clickhouse binary with keeper inside

Fixed:
- option (1) didn't work since docker_compose_keeper.yaml didn't create
target clickhouse-keeper at all
- if clickhouse-keeper existed, option (1) was taken but
  clickhouse-keeper could be just a link to clickhouse binary (the link
  is created always during build if cmake option BUILD_STANDALONE_KEEPER is OFF)
 2022-09-09 14:51:34 +00:00
robot-clickhouse
aeac3916ef Automatic style fix 2022-09-09 12:27:13 +00:00
Antonio Andelic
32483aeec8 Merge branch 'master' into keeper-storage 2022-09-09 12:24:01 +00:00
Sergei Trifonov
76eb001e78 add test for changeable_in_readonly + fix SQL in docs 2022-09-09 14:01:29 +02:00
Alexander Tokmakov
48927ba0ac
Merge branch 'master' into no-hardlinks-while-making-backup-of-mergetree-in-atomic-db 2022-09-09 14:24:44 +03:00
Vitaly Baranov
10629a66e5 Fix black. 2022-09-08 20:58:51 +02:00
Vitaly Baranov
e48cd2b6f6 Add more test cases. 2022-09-08 18:14:25 +02:00
robot-clickhouse
807b09f6b0 Automatic style fix 2022-09-08 15:52:00 +00:00
Sergei Trifonov
f92b6a6176
Merge branch 'master' into readonly-settings-allow 2022-09-08 17:44:32 +02:00
Sergei Trifonov
62541ab764 fix more tests and clang tidy build 2022-09-08 17:43:09 +02:00
Antonio Andelic
b99996961d Address PR comments 2022-09-08 12:45:36 +00:00
Vitaly Baranov
122009a2bd Use table lock if database is ordinary and zero-copy-replication is enabled. 2022-09-08 13:54:59 +02:00
Robert Schulze
c16707ff00
chore: delete obsolete modelEvaluate() function + SYSTEM.MODELS view 
- The deleted function modelEvaluate() was superseded by
  catboostEvaluate().

- Also delete the external model repository, as modelEvaluate() was it's
  last user. Additionally remove the system view SYSTEM.MODELS for
  inspecting the repository.

- SYSTEM RELOAD MODELS is also obsolete. HOWEVER, it was retained and
  made a no-op instead of deleted.

  Why?
  The reason is that RBAC in distributed setups works by storing
  privileges (granted and revoked) as plain SQL statements in Keeper.
  Nodes read these statements at startup and parse them. If a privilege
  for SYSTEM RELOAD MODELS exists but parser doesn't recognize it
  nodes would fail to come up.

  Considered but rejected alternatives:
  - Ignore SYSTEM RELOAD MODELS during parsing RBAC privileges and
    return an error for regular SYSTEM RELOAD MODELS SQL. Special-case
    of no-op behavior, too brittle.
  - Remove SYSTEM RELOAD MODELS manually from Keeper via command-line
    manipulation of Keeper nodes or via SQL by dropping the privileges.
    Needs user intervention during upgrade.
 2022-09-08 09:10:11 +00:00
Robert Schulze
60f9f6855d
feat: implement catboost in library-bridge 
This commit moves the catboost model evaluation out of the server
process into the library-bridge binary. This serves two goals: On the
one hand, crashes / memory corruptions of the catboost library no longer
affect the server. On the other hand, we can forbid loading dynamic
libraries in the server (catboost was the last consumer of this
functionality), thus improving security.

SQL syntax:

  SELECT
    catboostEvaluate('/path/to/model.bin', FEAT_1, ..., FEAT_N) > 0 AS prediction,
    ACTION AS target
  FROM amazon_train
  LIMIT 10

Required configuration:

  <catboost_lib_path>/path/to/libcatboostmodel.so</catboost_lib_path>

*** Implementation Details ***

The internal protocol between the server and the library-bridge is
simple:

- HTTP GET on path "/extdict_ping":
  A ping, used during the handshake to check if the library-bridge runs.

- HTTP POST on path "extdict_request"
  (1) Send a "catboost_GetTreeCount" request from the server to the
      bridge, containing a library path (e.g /home/user/libcatboost.so) and
      a model path (e.g. /home/user/model.bin). Rirst, this unloads the
      catboost library handler associated to the model path (if it was
      loaded), then loads the catboost library handler associated to the
      model path, then executes GetTreeCount() on the library handler and
      finally sends the result back to the server. Step (1) is called once
      by the server from FunctionCatBoostEvaluate::getReturnTypeImpl(). The
      library path handler is unloaded in the beginning because it contains
      state which may no longer be valid if the user runs
      catboost("/path/to/model.bin", ...) more than once and if "model.bin"
      was updated in between.
  (2) Send "catboost_Evaluate" from the server to the bridge, containing
      the model path and the features to run the interference on. Step (2)
      is called multiple times (once per chunk) by the server from function
      FunctionCatBoostEvaluate::executeImpl(). The library handler for the
      given model path is expected to be already loaded by Step (1).

Fixes #27870
 2022-09-08 09:01:32 +00:00
Vitaly Baranov
31ed722572
Merge pull request #41044 from vitlibar/more-conventional-conversion-yaml-to-xmk 
More conventional conversion yaml to xml
 2022-09-07 13:46:32 +02:00
Vitaly Baranov
9c847ceec9 No hardlinks while making backup of MergeTree in atomic database. 2022-09-07 11:44:50 +02:00
Daniil Rubin
f75376b7f8 Apply black formatter to tests 2022-09-07 11:05:40 +03:00
Daniil Rubin
4cc397da84 Add tests for deltalake storage 2022-09-07 11:05:40 +03:00
Daniil Rubin
0e9c3f299c Fix bug in test 2022-09-07 11:05:40 +03:00
Daniil Rubin
7d8cc20c46 Add select test 2022-09-07 11:05:40 +03:00
Daniil Rubin
6b95faf2b3 Add createquery test 2022-09-07 11:05:40 +03:00
Daniil Rubin
3e87fd0ee4 Add test template 2022-09-07 11:05:40 +03:00
Vitaly Baranov
63e992d52d Edit test configs. 2022-09-06 17:09:26 +02:00
Sergei Trifonov
6217559175
Merge branch 'master' into readonly-settings-allow 2022-09-06 16:10:48 +02:00
Antonio Andelic
5eb034a920 Merge branch 'master' into start-embedded-keeper-async 2022-09-06 12:14:47 +00:00
Antonio Andelic
4a68bfef39 Fix tests with async Keeper start 2022-09-06 12:12:17 +00:00
Sergei Trifonov
f77809ddbc
Merge pull request #40900 from ClickHouse/s3-detailed-metrics 
S3 detailed metrics
 2022-09-06 13:20:57 +02:00
Anton Ivashkin
e91d22a5b7 Merge branch 'fix-ordinary-system-unfreeze' of github.com:PolyProgrammist/ClickHouse into fix-ordinary-system-unfreeze 2022-09-06 12:18:05 +03:00
Anton Ivashkin
5e7cf1f238 Fix test_merge_tree_s3/test.py::test_freeze_system_unfreeze[node] 2022-09-06 12:14:40 +03:00
Alexey Milovidov
59bd94dcf8
Merge pull request #40975 from vitlibar/fix-access-rights-for-describe 
Fix access rights for DESCRIBE TABLE
 2022-09-06 02:38:34 +03:00
Alexey Milovidov
4c30dbc905
Merge pull request #40968 from ClickHouse/s3-sharding-2 
Sharding s3 key names (2)
 2022-09-05 22:47:25 +03:00
ianton-ru
39e1fc7a0f
Merge branch 'master' into fix-ordinary-system-unfreeze 2022-09-05 17:10:59 +03:00
Sergei Trifonov
1c247308f1
Merge branch 'master' into s3-detailed-metrics 2022-09-05 15:55:41 +02:00
Sergei Trifonov
225773634c fix integration test for profile events s3 2022-09-05 15:54:28 +02:00
Kseniia Sumarokova
cff1dd6a8e
Merge branch 'master' into move-strange-test 2022-09-05 15:19:57 +02:00
Antonio Andelic
e07539d1b9 Merge branch 'master' into keeper-storage 2022-09-05 09:54:13 +00:00
Antonio Andelic
3a0581e990
Merge pull request #40543 from Lloyd-Pottiger/feat/support-read-only-for-embeddedrocksdb 
Add read-only support for EmbeddedRocksDB
 2022-09-05 09:31:07 +02:00
Alexey Milovidov
cda8052df0 Fix trash 2022-09-05 03:46:10 +02:00
Alexey Milovidov
e70d526896 Fix trash 2022-09-05 03:35:43 +02:00
Alexey Milovidov
bff96b0af6
Merge branch 'master' into fix-access-rights-for-describe 2022-09-05 04:34:57 +03:00
Alexey Milovidov
cc50aa39a2 Merge branch 's3-sharding-2' of github.com:ClickHouse/ClickHouse into s3-sharding-2 2022-09-05 02:25:01 +02:00
Alexey Milovidov
79e3723272 Merge branch 'master' into s3-sharding-2 2022-09-05 02:24:35 +02:00
robot-clickhouse
89d40b6f70 Automatic style fix 2022-09-05 00:23:36 +00:00
Alexey Milovidov
a0b5567d3e Move strange test 2022-09-05 02:16:48 +02:00
Alexey Milovidov
007680d93f Preparation 2022-09-05 02:13:51 +02:00
Vitaly Baranov
6c164905ca Add test. 2022-09-04 18:57:51 +02:00
Alexey Milovidov
c4adc9ed8f Remove trash 2022-09-04 04:28:08 +02:00
Alexey Milovidov
13a129bee7 Merge branch 'master' into remove-useless-line 2022-09-04 03:24:40 +02:00
Lloyd-Pottiger
ec68ed8a93 fix test 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-09-03 10:44:07 +08:00
Sergei Trifonov
005b2588d8
Merge branch 'master' into readonly-settings-allow 2022-09-02 16:37:43 +02:00
Lloyd-Pottiger
b76af9284e style check 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-09-02 19:29:57 +08:00
Lloyd-Pottiger
f499381f57 move functional test to integration test 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-09-02 18:45:37 +08:00
Antonio Andelic
9ca20b97c2
Merge branch 'master' into keeper-storage 2022-09-02 12:13:07 +02:00
Lloyd-Pottiger
5a6b2106b5 Merge branch 'master' of github.com:ClickHouse/ClickHouse into feat/support-read-only-for-embeddedrocksdb 2022-09-02 18:10:00 +08:00
robot-clickhouse
46f74fcbaf Automatic style fix 2022-09-02 07:49:09 +00:00
Antonio Andelic
b02bf49939 Use path prefix 2022-09-02 07:34:26 +00:00
Lloyd-Pottiger
59dccd6e49 fix test 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-09-02 11:14:49 +08:00
Sergei Trifonov
7fec55eea4 work in progress 2022-09-02 04:12:05 +02:00
Frank Chen
9d63cbe811 Merge 'origin/master' into tracing_context_propagation to resolve conflicts 2022-09-01 23:18:59 +08:00
Vitaly Baranov
6dcca686cb
Merge pull request #40827 from vitlibar/fix-incremental-backups-for-log-family 
Fix incremental backups for Log family.
 2022-09-01 13:13:32 +02:00
Sema Checherinda
7b59fdc042
Merge pull request #40779 from CheSema/detached-parts-metric 
Metric for the number of detached parts
 2022-09-01 12:24:42 +02:00
Vitaly Baranov
007ae0e6cc Fix incremental backups for Log family. 2022-08-31 12:57:28 +02:00
Antonio Andelic
9dd1a9859d Merge branch 'master' into keeper-storage 2022-08-31 10:12:11 +00:00
root
ea517de61a style check correction 2022-08-30 23:20:19 -03:00
root
f02f7ce99c addressed minor change requests 2022-08-30 23:20:19 -03:00
root
84a617675d style check correction in test.py 2022-08-30 23:20:19 -03:00
root
3ced3c9933 style check correction in test.py 2022-08-30 23:20:19 -03:00
root
1d29a494db Modified test code to cover various scenarios for custom keys 2022-08-30 23:20:19 -03:00
root
d5db88fa27 Structured-logging-custom-keys PR init 2022-08-30 23:20:19 -03:00
Sema Checherinda
9bb4f556c3 work on remarks on the review 2022-08-30 20:18:35 +02:00
Antonio Andelic
3c098bd0cb Add integration test for KeeperMap 2022-08-30 13:41:57 +00:00
Alexander Tokmakov
022f440ad0
Merge pull request #40769 from arthurpassos/caresptrresolver-hosts-patch 
Fix CaresPTRResolver not reading hosts file
 2022-08-30 14:35:10 +03:00
alesapin
2588901bc9
Merge pull request #40780 from ClickHouse/add_cache_to_zero_copy_test 
Add cache to zero copy replication test
 2022-08-30 12:42:05 +02:00
Kseniia Sumarokova
c88db2ef97
Merge pull request #40751 from kssenii/fix-mysql-timeouts 
Fix issue with mysql db / table function timeouts
 2022-08-30 11:59:01 +02:00
Frank Chen
f17d56b528 Merge branch 'master' into tracing_context_propagation 2022-08-30 14:24:36 +08:00
Sema Checherinda
e436b4f4cc Metric for the number of detached parts 2022-08-30 02:12:57 +02:00
alesapin
ac41de73df Add cache to zero copy replication test 2022-08-30 00:17:01 +02:00
Kseniia Sumarokova
8001560082
Fix test 2022-08-30 00:14:00 +02:00
Vitaly Baranov
33f72fb011
Merge pull request #40060 from ClickHouse/vitlibar-increase-timeout-for-test_concurrent_backups 
Increase timeout for test_concurrent_backups
 2022-08-29 22:25:56 +02:00
Arthur Passos
dd49b44abb Fix host_regexp hosts file tst 2022-08-29 15:58:18 -03:00
Arthur Passos
961365c7a4 Fix CaresPTRResolver not reading hosts file 2022-08-29 15:11:39 -03:00
Kseniia Sumarokova
c5c48e44ea
Merge branch 'master' into fix-mysql-timeouts 2022-08-29 19:33:29 +02:00
kssenii
0a6c4b9265 Fix 2022-08-29 16:20:53 +02:00
Vladimir C
5cbe7e0846
Merge pull request #40548 from ClickHouse/vdimir/warn-suppress-40330 
Add config option warning_supress_regexp
 2022-08-29 14:02:00 +02:00
Alexander Tokmakov
8d90d30d37
Merge pull request #40589 from ClickHouse/remove_wrong_code_from_mutations 
Remove wrong code for skipping mutations in MergeTree
 2022-08-29 14:18:59 +03:00
Alexander Tokmakov
eda0582ec0
Merge pull request #40641 from ClickHouse/fix_startup_of_dropped_replica 
Do not try to strartup dropped replica
 2022-08-29 14:15:15 +03:00
Vitaly Baranov
2bec3d3a7c Increase timeout for test_concurrent_backups 2022-08-29 13:13:43 +02:00
alesapin
7ce0afc0df
Merge pull request #40670 from Avogar/kafka 
Add setting to disable limit on kafka_num_consumers
 2022-08-29 10:53:35 +02:00
Alexey Milovidov
b72fceb441
Merge pull request #40708 from lesandie/test_s3_table_functions 
Added integration test for s3 table function
 2022-08-29 03:49:53 +03:00
Alexey Milovidov
f462160bd1 Merge remote-tracking branch 'origin/master' into remove-useless-line 2022-08-28 21:00:29 +02:00
Diego Nieto (lesandie)
3d50dbea34 Black reformatting 2022-08-28 20:05:39 +02:00
Alexey Milovidov
fc29303ff3
Merge pull request #40685 from ClickHouse/nlp-functional-test 
Rewrite NLP tests from integration to functional
 2022-08-28 17:05:42 +03:00
Alexey Milovidov
f518844f06
Merge pull request #40709 from ClickHouse/all_new_table_functions_must_be_documented 
All new table functions must be documented
 2022-08-28 17:05:20 +03:00
Alexey Milovidov
18c56b722f Fix integration test 2022-08-28 04:30:27 +02:00
Alexey Milovidov
b42d26acfe Remove one line from XML, because we do not care 2022-08-28 02:44:02 +02:00
Alexey Milovidov
5b57c91ad6 Rename table function MeiliSearch to meilisearch 2022-08-28 02:08:51 +02:00
Diego Nieto (lesandie)
9a07489151 Added integration test for s3 table function 2022-08-27 18:33:04 +02:00
Alexey Milovidov
6414a01987 Rewrite NLP tests from integration to functional 2022-08-26 22:36:34 +02:00
Kruglov Pavel
d4d0ec1c95
Fix style 2022-08-26 16:44:54 +02:00
Alexander Tokmakov
7fa0cc02f2 fix tests 2022-08-26 15:33:04 +02:00
avogar
b8061532ac Add setting to disable limit on kafka_num_consumers 2022-08-26 12:49:39 +00:00
Kseniia Sumarokova
47d48a7b5c
Merge pull request #40579 from kssenii/add-test 
Add test for drop fs cache on cluster
 2022-08-26 12:55:13 +02:00
Lloyd-Pottiger
0d33327eda
Merge branch 'master' into feat/support-read-only-for-embeddedrocksdb 2022-08-26 15:36:21 +08:00
Kseniia Sumarokova
e2014c9872
Update test.py 2022-08-25 23:01:31 +02:00
Alexander Tokmakov
eae2667a1c
Merge pull request #40148 from ClickHouse/replicate-tables-readonly 
`ReplicatedMergeTree` tables as readonly until attach is finished
 2022-08-25 19:39:19 +03:00
Kseniia Sumarokova
e46d34c722
Update test.py 2022-08-25 18:37:25 +02:00
Alexander Tokmakov
a6f55b1430 fix 2022-08-25 16:27:12 +02:00
robot-clickhouse
6e538f04c4 Automatic style fix 2022-08-25 12:30:57 +00:00
Alexander Tokmakov
0aea4c7184 fix 2022-08-25 14:22:27 +02:00
alesapin
7d1e3ecfa3
Merge pull request #40506 from ClickHouse/fix_bad_exception 
Fix lost errors in AWS CPP SDK
 2022-08-25 13:07:35 +02:00
alesapin
c4f8811701
Merge pull request #40395 from ClickHouse/data_source_description 
Add copy shortcut for backups if dest and source support it
 2022-08-25 12:36:30 +02:00
Kseniia Sumarokova
73bb5a6b9e
Update test.py 2022-08-25 12:10:46 +02:00
robot-clickhouse
669ae365c1 Automatic style fix 2022-08-25 11:00:01 +02:00
vdimir
8bc29abdfa Test warning_supress_regexp in test_log_levels_update 2022-08-25 11:00:01 +02:00
Frank Chen
99c37ce6c6
Merge branch 'master' into tracing_context_propagation 2022-08-25 10:07:16 +08:00
Kseniia Sumarokova
bfc3180c9e
Update test.py 2022-08-25 00:20:50 +02:00
Alexey Milovidov
aaafb797c0
Merge pull request #40541 from ClickHouse/paranoid_check_for_s3_uploads 
Add paranoid option to check blobs to S3 after upload
 2022-08-24 23:51:22 +02:00
Alexander Tokmakov
3cd26aafe4 remove wrong code for skipping mutations in MergeTree 2022-08-24 20:58:59 +02:00
Antonio Andelic
da0d6056f4 Merge branch 'master' into replicate-tables-readonly 2022-08-24 17:29:33 +00:00
Lloyd-Pottiger
9d7ace3286
Merge branch 'master' into feat/support-read-only-for-embeddedrocksdb 2022-08-24 23:52:34 +08:00
Lloyd-Pottiger
dc64341876 add tests 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-08-24 23:42:13 +08:00
Alexander Tokmakov
f68db41c1c
Merge pull request #40575 from Avogar/try-fix-s3-test 
Fix flaky schema inference cache tests
 2022-08-24 18:24:05 +03:00
alesapin
eeb0a3584e Merge branch 'master' into data_source_description 2022-08-24 15:26:18 +02:00
Kseniia Sumarokova
6ac35dabcf
Update test.py 2022-08-24 15:03:02 +02:00
kssenii
1033c14859 Add test 2022-08-24 15:01:21 +02:00
alesapin
f0b733313f Merge branch 'master' into fix_bad_exception 2022-08-24 14:44:40 +02:00
alesapin
cac3005f4d Merge branch 'master' into paranoid_check_for_s3_uploads 2022-08-24 14:43:11 +02:00
Alexander Tokmakov
f9f85a0e8b Revert "Parallel distributed insert select from *Cluster table functions (#39107)" 
This reverts commit d3cc234986.
 2022-08-24 15:17:15 +03:00
avogar
93d0113b14 Fix flacky schema inference cache tests 2022-08-24 11:28:56 +00:00
Alexander Tokmakov
c1d933b746
Update test.py 2022-08-24 13:39:45 +03:00
alesapin
dc95d86d6e
Merge pull request #40064 from ClickHouse/keeper-force-recovery-more-retries 
Add retries for more commands in `test_keeper_force_recovery`
 2022-08-23 15:44:28 +02:00
robot-clickhouse
f0b18a1082 Automatic style fix 2022-08-23 13:21:29 +00:00
alesapin
07286dd042 Add paranoid option to check blobs to S3 after upload 2022-08-23 15:12:09 +02:00
alesapin
7c7fa9472f One more comment 2022-08-23 14:07:50 +02:00
alesapin
2f478774e5 Add comment to proxy 2022-08-23 14:05:05 +02:00
Vadim Volodin
9c371d518f fix enable_unfreeze handler and sanitizer hints 2022-08-23 13:53:06 +03:00
alesapin
802d56df49 Merge branch 'master' into fix_bad_exception 2022-08-23 12:22:05 +02:00
Antonio Andelic
0765495be1
Merge pull request #39986 from Lloyd-Pottiger/feat/support-ttl-for-embeddedrocksdb 
Support TTL for EmbeddedRocksDB
 2022-08-23 08:27:47 +02:00
robot-clickhouse
315d38c365 Automatic style fix 2022-08-22 22:43:00 +00:00
alesapin
780f39b108 Fix lost errors in AWS CPP SDK 2022-08-23 00:34:39 +02:00
Lloyd-Pottiger
85a1a36167 remove read_only option 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-08-22 23:31:55 +08:00
alesapin
b4d6afd5b3 Black 2022-08-22 13:51:48 +02:00
alesapin
704d7fdc41 Fix copy and use disks 2022-08-21 18:18:35 +02:00
Antonio Andelic
76ecb28daf Increase timeouts 2022-08-20 11:52:10 +00:00
Frank Chen
024fef1aec
Merge branch 'master' into tracing_context_propagation 2022-08-20 14:23:36 +08:00
Azat Khuzhin
b92d8b302a tests: fix test_storage_s3::test_wrong_format_usage flakiness 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-19 21:51:23 +02:00
Sergei Trifonov
7c18d5b34f
Merge pull request #40348 from ClickHouse/threads-soft-limit-rename 
split concurrent_threads_soft_limit setting into two
 2022-08-19 13:39:32 +02:00
Kruglov Pavel
c4f9508b80
Merge pull request #40355 from Avogar/better-schema-cache-test 
Make test for s3 schema inference cache better
 2022-08-19 12:17:49 +02:00
Antonio Andelic
5e2c8869a9 Merge branch 'master' into replicate-tables-readonly 2022-08-19 09:41:11 +00:00
Antonio Andelic
3f7d28cc6b Address PR comments 2022-08-19 09:39:59 +00:00
Lloyd-Pottiger
51f53af482 fix tests 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-08-19 15:24:00 +08:00
avogar
67f848a428 Fix style 2022-08-18 17:15:30 +00:00
avogar
1fb99f51b0 Make test for s3 schema inference cache better 2022-08-18 16:50:59 +00:00
Frank Chen
a3b6ad2a65
Merge branch 'master' into tracing_context_propagation 2022-08-18 20:59:07 +08:00
Sergei Trifonov
42a711015f split concurrent_threads_soft_limit setting into two: concurrent_threads_soft_limit_{num,ratio_to_cores} 2022-08-18 14:51:08 +02:00
Nikolai Kochetov
5a85531ef7
Merge pull request #38286 from Avogar/schema-inference-cache 
Add schema inference cache for s3/hdfs/file/url
 2022-08-18 13:07:50 +02:00
Kseniia Sumarokova
d8fd1f9c37
Merge pull request #36171 from kssenii/make-cache-composable 
Make cache composable, allow not to evict certain files (regarding .idx, .mrk, ..), delete old cache version
 2022-08-18 12:37:34 +02:00
Vladimir C
7ac0eb1af1
Merge pull request #40035 from ClickHouse/vdimir/cluster-discovery-observer 
Add observer mode to (zoo)keeper cluster discovery feature
 2022-08-18 10:53:49 +02:00
Lloyd-Pottiger
340c8d1c18 address comments 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-08-17 23:27:18 +08:00
Lloyd-Pottiger
19c4385881
Merge branch 'master' into feat/support-ttl-for-embeddedrocksdb 2022-08-17 22:17:10 +08:00
kssenii
d63d34f7bb Merge master 2022-08-17 15:49:32 +02:00
avogar
4c64b7a69e Try fix s3 test 2022-08-17 12:25:58 +00:00
avogar
8dd54c043d Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-cache 2022-08-17 11:47:40 +00:00
Alexander Tokmakov
caa270b72a
Merge pull request #39933 from ClickHouse/auto_convert_ordinary_to_atomic 
Add flag that enables automatic conversion from Ordinary to Atomic
 2022-08-17 12:40:49 +03:00
Nikita Mikhaylov
a7c3f23dfb
Merge branch 'master' into auto_convert_ordinary_to_atomic 2022-08-17 00:38:25 +02:00
Kruglov Pavel
afd5221d58
FIx check black 2022-08-16 18:53:55 +02:00
avogar
d5f7337ba3 Try fix tests 2022-08-16 14:57:19 +00:00
Alexander Tokmakov
832d7c6e7c Merge branch 'master' into replicated_database_improvements 2022-08-16 15:49:13 +02:00
Kruglov Pavel
169b38c878
Fix s3 test 2022-08-16 12:40:10 +02:00
Kruglov Pavel
795262f16c
Fix hdfs test 2022-08-16 12:39:35 +02:00
Antonio Andelic
2ed42bca7e Merge branch 'master' into replicate-tables-readonly 2022-08-16 08:17:53 +00:00
Alexander Tokmakov
1e6f7a7c52 make it work with python3.8 2022-08-16 09:35:45 +02:00
kssenii
eb26b219b9 Merge master 2022-08-16 00:56:27 +02:00
robot-clickhouse
1b11a5c6fe Automatic style fix 2022-08-15 20:19:41 +00:00
Alexander Tokmakov
26c2a0ce5e try to print stacktraces if query timeouts 2022-08-15 22:12:07 +02:00
avogar
c4ff3ffeea Rename settings 2022-08-15 12:45:18 +00:00
Nikita Mikhaylov
d3cc234986
Parallel distributed insert select from *Cluster table functions (#39107) 2022-08-15 12:41:17 +02:00
Alexey Milovidov
3a5f05bd22
Merge pull request #40175 from ClickHouse/disable-zero-copy-replication 
Disable zero-copy replication by default
 2022-08-14 12:47:38 +03:00
Alexey Milovidov
216c1d8efa
Revert "Use separate counter for RSS in global memory tracker." 2022-08-14 09:50:12 +03:00
Alexey Milovidov
b5e3d908e2 Fix tests 2022-08-14 04:11:17 +02:00
Alexey Milovidov
91042e9dd8 Update test configs 2022-08-14 01:43:24 +02:00
vdimir
89dca02e1d
Fix ClusterDiscovery 2022-08-12 13:26:17 +00:00
vdimir
e15f20e2e3
Update test_cluster_discovery/test.py 2022-08-12 13:26:16 +00:00
robot-clickhouse
66269cf6c9
Automatic style fix 2022-08-12 13:26:16 +00:00
vdimir
cef6c4f8ce
Add observer mode to (zoo)keeper cluster discovery feature 2022-08-12 13:26:12 +00:00
robot-clickhouse
479c945c3b Automatic style fix 2022-08-12 12:16:50 +00:00
Antonio Andelic
c6b3a897d2 Test replicated table without ZK, small fixes 2022-08-12 11:56:46 +00:00
kssenii
768a207176 Merge master 2022-08-11 22:25:53 +02:00
avogar
c0bb6303b9 Fix check black 2022-08-11 10:55:18 +00:00
Nikolai Kochetov
971d94e492
Merge pull request #38682 from ClickHouse/separate-counter-for-rss-in-memory-tracker 
Use separate counter for RSS in global memory tracker.
 2022-08-11 10:11:18 +02:00
Alexey Milovidov
8374f31306
Merge pull request #39425 from arenadata/ADQM-485 
Add support of dates from year 1900 to 2299 for Date32 and DateTime64
 2022-08-11 05:01:53 +03:00
Anton Popov
3fdf428834
Merge pull request #39186 from Avogar/numbers-schema-inference 
Add new features in schema inference
 2022-08-11 00:53:54 +02:00
Nikolay Degterinsky
496568101f
Merge pull request #39277 from ClibMouse/Structured-Logging-Support 
Structured logging support
 2022-08-10 22:57:48 +02:00
Nikita Mikhaylov
8e8e845a89
Merge branch 'master' into separate-counter-for-rss-in-memory-tracker 2022-08-10 16:44:24 +02:00
Lloyd-Pottiger
5488a392ab address comments 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-08-10 21:20:25 +08:00
Antonio Andelic
d8f3511b9e add more retries 2022-08-10 12:39:20 +00:00
kssenii
5c3227ba56 Merge master 2022-08-10 12:00:34 +02:00
Sema Checherinda
de49e1ae9a
Merge pull request #40007 from CheSema/no-ip-for-certs 
remove using fixed IP from tests with SSL
 2022-08-09 23:06:37 +02:00
Mallik Hassan
666fb8dc42
Merge branch 'ClickHouse:master' into Structured-Logging-Support 2022-08-09 15:39:00 -03:00
Nikolai Kochetov
fe6533f41d Disable test for sanitizers. 2022-08-09 12:25:58 +00:00
Kruglov Pavel
088e8cf9bd
Merge branch 'master' into numbers-schema-inference 2022-08-09 14:00:36 +02:00
Sema Checherinda
b3df5fe951
Update test.py 2022-08-09 12:52:16 +02:00
Sema Checherinda
59cb67c1f6
Update test.py 2022-08-09 12:50:54 +02:00
Sema Checherinda
d5c7cd0ef3
Update test.py 2022-08-09 12:50:28 +02:00
Sema Checherinda
070a39543b
Update tests/integration/test_tlsv1_3/test.py 
fix comment

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2022-08-09 12:48:31 +02:00
Sema Checherinda
4956e9dc49
Update parallel_skip.json 
no comma at the end of file
 2022-08-09 12:47:19 +02:00
Sema Checherinda
fd0aff444a regenerate certificates 2022-08-09 12:02:23 +02:00
Sema Checherinda
8dd6540dac reenable parallel execution for test_grpc_protocol_ssl test_ssl_cert_authentication test_tlsv1_3 2022-08-09 12:01:38 +02:00
Sema Checherinda
2c84fcdc3b test_grpc_protocol_ssl test_ssl_cert_authentication test_tlsv1_3 don't use fixed IP for instances 2022-08-09 12:01:15 +02:00
Nikolai Kochetov
8a4c57f641 Added __init__.py 2022-08-08 16:41:11 +00:00
Alexander Tokmakov
2c5da1466d Merge branch 'master' into replicated_database_improvements 2022-08-08 16:08:27 +02:00
Alexander Tokmakov
848511affb take permanently detached tables into account 2022-08-08 16:06:08 +02:00
robot-clickhouse
f10d4f6c61 Automatic style fix 2022-08-08 13:46:38 +00:00
Nikolai Kochetov
540ae64f37 Fix big memory usage for fetch. 2022-08-08 13:32:49 +00:00
Robert Schulze
ad0d060dc1
Merge pull request #39904 from ClickHouse/library-bridge-refactoring 
Prepare library-bridge for catboost integration
 2022-08-08 12:15:01 +02:00
Alexey Milovidov
0921548a37
Merge pull request #39527 from pkit/pkit/fix_nats_tls 
fix nats-io TLS support
 2022-08-08 08:28:14 +03:00
Alexander Tokmakov
f2c4cad255 add flag that enables automatic canversion from Ordinary to Atomic 2022-08-05 21:41:25 +02:00
avogar
9b1a267203 Refactor, remove TTL, add size limit, add system table and system query 2022-08-05 16:20:15 +00:00
Nikolai Kochetov
c6e3e14bcc
Merge pull request #39184 from ClickHouse/respect-remote_url_allow_hosts-for-other-dict-sources 
Respect remote_url_allow_hosts in relevant dictionary sources.
 2022-08-05 17:25:53 +02:00
kssenii
91f4143b6e Merge master 2022-08-05 12:14:08 +02:00
Alexander Tokmakov
65e1ce8ad1
Merge pull request #39674 from tonickkozlov/tonickkozlov/fix-default-database-resolution 
Default database resolution in distributed reads
 2022-08-05 13:09:15 +03:00
alesapin
329120ca7a
Merge pull request #39755 from ClickHouse/keeper-create-snapshot-on-exit 
Create Keeper snapshot on exit
 2022-08-05 12:00:16 +02:00
Alexander Tokmakov
d9190a8121
Merge pull request #39872 from ClickHouse/store_cleanup_all_disks 
Cleanup unused dirs from `store/` on all disks
 2022-08-05 12:21:59 +03:00
Constantine Peresypkin
10c76917ea fix nats-io TLS support 
nats-io library needs `NATS_HAS_TLS` define to correctly compile-in TLS
support

fixes #39525
 2022-08-05 11:20:05 +02:00
kssenii
7a9b0bc47f Merge master 2022-08-05 01:48:52 +02:00
Robert Schulze
7df30747f3
Typos 2022-08-04 19:27:38 +00:00
Alexander Tokmakov
110b60e5b7 fix tests 2022-08-04 19:48:55 +02:00
Alexander Tokmakov
460f2f2e39 Merge branch 'master' into store_cleanup_all_disks 2022-08-04 19:48:41 +02:00
Alexander Tokmakov
ebbaed46bd Merge branch 'master' into tonickkozlov/fix-default-database-resolution 2022-08-04 15:20:44 +02:00
Antonio Andelic
07a85513e8 Merge branch 'master' into keeper-create-snapshot-on-exit 2022-08-04 13:12:26 +00:00
Ilya Yatsishin
cecdc313d8
Merge branch 'master' into integration-tests-7 2022-08-04 11:37:16 +02:00
Alexander Tokmakov
1133425624
Update storage_conf.xml 2022-08-04 12:16:38 +03:00
Vitaly Baranov
a952a5dfe5
Merge pull request #39859 from vitlibar/fix-flaky-test_async_backups_to_same_destination 
Fix flaky integration test test_async_backups_to_same_destination.
 2022-08-04 09:46:25 +02:00
Antonio Andelic
8094f33078 Add init file for test 2022-08-04 06:48:33 +00:00
Alexey Milovidov
9e46abc560
Merge pull request #39420 from amosbird/better-projection1-fix1 
Normalize AggregateFunction types and state representations
 2022-08-04 03:06:55 +03:00
Alexander Tokmakov
a269cf29e4 remove unused dirs from store on all disks 2022-08-03 19:49:16 +02:00
Vitaly Baranov
dc25f18f13 Fix flaky integration test test_async_backups_to_same_destination. 2022-08-03 14:22:09 +02:00