thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 03:12:43 +00:00
Code Issues Packages Projects Releases Wiki Activity
117,551 Commits 9,326 Branches 1,783 Tags 2.1 GiB
241d711264
Commit Graph

5955 Commits

Author SHA1 Message Date
Sema Checherinda
a1ee7d52e1
Merge pull request #49432 from CheSema/lost-blobs 
all s3-blobs removed when merge aborted, remove part from failed fetch without unlock keper
 2023-05-12 13:19:27 +02:00
Sema Checherinda
5706e05757 adding more comments 2023-05-12 13:00:56 +02:00
Nikita Taranov
536c5ac261
Merge branch 'master' into optimize_reading2 2023-05-12 12:59:32 +02:00
Sema Checherinda
66e3712dca
Merge branch 'master' into lost-blobs 2023-05-11 08:23:25 +02:00
Sema Checherinda
9724688608 rename field, do not fetch PreActive 2023-05-11 08:22:37 +02:00
Azat Khuzhin
d56e1423ef
Merge branch 'master' into build/llvm-16 2023-05-10 21:23:26 +02:00
Sema Checherinda
7fbf87be17 rework WriteBufferFromS3, squashed 2023-05-10 18:31:47 +00:00
Alexander Gololobov
049ef9dda5
Merge branch 'master' into fix_lwd_and_object 2023-05-10 19:54:26 +02:00
Nikita Taranov
ec63a6ee7c
Merge branch 'master' into optimize_reading2 2023-05-10 17:24:37 +02:00
Alexey Milovidov
4a4eb5b171
Merge pull request #49660 from CurtizJ/fix-sparse-columns-reload 
Fix reading from sparse columns after restart
 2023-05-10 18:08:22 +03:00
Alexey Milovidov
33a58aacea
Merge pull request #49719 from ClickHouse/make-projections-non-experimental 
Make projections production-ready
 2023-05-10 18:07:02 +03:00
Azat Khuzhin
a0cc5b7e47 Merge remote-tracking branch 'u/master' into build/llvm-16 2023-05-10 16:28:36 +02:00
Alexander Gololobov
e6f4e5cd10 Ignore LWD column in checkPartDynamicColumns 2023-05-10 16:10:04 +02:00
Han Fei
d17aa828b3
Merge pull request #49361 from hanfei1991/hanfei/fiu 
support named fault injection and add a simple test
 2023-05-10 15:51:51 +02:00
Roman Vasin
632c0eeefc
Add support of temporary tables in FROM part of ATTACH|REPLACE PARTITION FROM (#49436) 
* Add temp tables after FROM

* Resolve works fine with temp and non-temp tables

* Add usage of temporary table in  00626_replace_partition_from_table test

* Update documentation

* Add tests for ReplicatedMergeTree

* Move tests into 02731_replace_partition_from_temporary_table
 2023-05-10 15:25:12 +03:00
Alexey Milovidov
8a6e07f0ea Make projections production-ready 2023-05-10 03:35:13 +02:00
Anton Popov
996fcfe120 fix column ttl with sparse columns 2023-05-09 21:37:43 +00:00
Alexander Tokmakov
6f53ee289d fix another issue, add a test 2023-05-09 23:26:56 +02:00
Azat Khuzhin
d03ae2abfa Fix modernize-loop-convert clang-tidy warning 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-09 21:19:37 +02:00
Alexey Milovidov
a8bdb20fc4
Merge pull request #48787 from ClickHouse/background-memory-tracker 
Add MemoryTracker for the background tasks [Resubmit]
 2023-05-09 07:58:36 +03:00
Alexey Milovidov
9a73a04fe3
Merge pull request #42587 from ClickHouse/compress-marks-primary-key-by-default 
Compress marks and primary key by default
 2023-05-09 06:43:10 +03:00
Han Fei
64b67b9a37 Merge branch 'master' into hanfei/fiu 2023-05-09 01:14:30 +02:00
Alexander Tokmakov
aa829c4ddc better partitions hint in mutations finalization 2023-05-09 00:23:21 +02:00
Han Fei
82ce47ce56 address comments 2023-05-08 23:02:36 +02:00
Anton Popov
0612f5d782 fix sparse columns after reload 2023-05-08 11:50:21 +00:00
Alexey Milovidov
3cf1da7bf3
Merge branch 'master' into deprecate-in-memory-parts 2023-05-08 01:38:00 +03:00
Alexander Tokmakov
91489243c9 provide better partitions hint for merge selecting task 2023-05-08 00:22:07 +02:00
Alexander Tokmakov
29bbde85a6 split selectPartsToMerge into smaller functions 2023-05-07 22:06:25 +02:00
alesapin
33e691ad51 Better name 2023-05-07 14:26:21 +02:00
alesapin
a2ed457219 Better concurrent parts removal with zero copy 2023-05-07 14:02:24 +02:00
Alexey Milovidov
3b6edbacab Update tests 2023-05-07 03:45:17 +02:00
Alexey Milovidov
de3e66385c Merge branch 'master' into compress-marks-primary-key-by-default 2023-05-07 02:40:25 +02:00
Sema Checherinda
08bfc8f58d inserted parts has exlusive blobs 2023-05-07 01:51:43 +02:00
Sema Checherinda
f302c1986a use flag has_exclusive_blobs to track the part's originate 2023-05-06 03:07:29 +02:00
Alexey Milovidov
75a2589a42
Merge branch 'master' into deprecate-in-memory-parts 2023-05-06 00:49:59 +03:00
Robert Schulze
3dfc0bd265
Merge pull request #49413 from azat/build/headers 
Slightly reduce inter-header dependencies
 2023-05-05 23:37:58 +02:00
Nikolay Degterinsky
7a727bbbe3
Merge pull request #49520 from IvanTakarlikov-st/it-grammar-fixes 
Fix some grammar mistakes in documentation, code and tests
 2023-05-05 15:45:58 +02:00
Sema Checherinda
9a0bc891e4 fix can_remove_callback for Temporary parts 2023-05-04 19:39:16 +02:00
Ivan Takarlikov
8873856ce5 Fix some grammar mistakes in documentation, code and tests 2023-05-04 13:35:18 -03:00
Anton Popov
e61d7baa54 Merge remote-tracking branch 'upstream/master' into HEAD 2023-05-04 14:03:53 +00:00
Alexey Milovidov
261907c143 Merge branch 'master' into deprecate-in-memory-parts 2023-05-04 01:07:38 +02:00
alesapin
610e63bfd2 Fix build 2023-05-03 23:54:24 +02:00
wangxiaobo
2924837c7e
Fix Attaching a table with non-existent ZK path does not increase the ReadonlyReplica metric (#48954) 
* Attaching a table with non-existent ZK path does not increase the ReadonlyReplica metric bug fix.

* add integration test.

* add integration test.

* Update test.py

* format code

* format code
 2023-05-03 22:41:20 +03:00
alesapin
913b63edc9 Fix another zero copy bug 2023-05-03 21:28:33 +02:00
Anton Popov
c9705d6bf1
Merging #49066 (Better error handling during loading of parts) (#49430) 
* catch filesystem error when loading parts

* better exception handling during part loading

---------

Co-authored-by: cangyin <excangyin@gmail.com>
Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
 2023-05-03 19:10:06 +02:00
Nikita Mikhaylov
954e3b724c
Speedup outdated parts loading (#49317) 2023-05-03 18:56:45 +02:00
Anton Popov
7896d30737
Merge pull request #49122 from CurtizJ/add-async-insert-mt-setting 
Add `MergeTree` setting `async_insert`
 2023-05-03 16:01:13 +02:00
Nikita Taranov
30d216f863 fix 2023-05-03 13:49:46 +00:00
alesapin
6f3f202f7c
Merge branch 'master' into fix_zero_copy_not_atomic 2023-05-03 12:53:35 +02:00
Alexey Milovidov
f449df85b6 Deprecate in-memory parts 2023-05-03 00:31:09 +02:00
Azat Khuzhin
2b21b83979 Reduce inter-header dependencies 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-02 21:15:18 +02:00
Nikita Taranov
67bd861bc3 Merge branch 'master' into optimize_reading2 2023-05-02 14:23:56 +00:00
SmitaRKulkarni
997210c782
Merge pull request #48515 from ClickHouse/Clear_marks_cache_for_outdated_parts 
MergeTreeMarksLoader holds DataPart instead of DataPartStorage
 2023-05-02 15:11:14 +02:00
alesapin
24a744db4c Merge branch 'master' into fix_zero_copy_not_atomic 2023-05-02 14:57:55 +02:00
Nikita Mikhaylov
aa4c5fe958
Enhancements for background merges (#49313) 2023-05-02 13:43:59 +02:00
Anton Popov
bcdd2307fb add comments 2023-05-02 11:30:50 +00:00
Antonio Andelic
f695040a19
Merge pull request #48897 from ClickHouse/keeper-check-not-exists 
Add `CheckNotExists` request to Keeper
 2023-05-02 13:14:22 +02:00
Han Fei
d608ba41e7 try to fix 2023-05-02 00:05:36 +02:00
alesapin
0f6a4b973e
Merge branch 'master' into ttl_on_insert 2023-05-01 21:02:04 +02:00
alesapin
a3f9a8b87b Fix bug in removal of existing part directory 2023-05-01 20:32:01 +02:00
Nikita Taranov
979b68a488 fix 2023-05-01 18:24:30 +00:00
Han Fei
4d18a4f8d3
Merge branch 'master' into hanfei/fiu 2023-05-01 17:51:41 +02:00
Han Fei
b701293d46 support named fault injection and add a simple test 2023-05-01 17:44:37 +02:00
alesapin
66847386df
Merge pull request #49146 from ClickHouse/don_t_write_version_file_on_start 
Don't add metadata_version file if it doesn't exist
 2023-05-01 17:04:47 +02:00
Anton Popov
36d53e0d6c
Merge branch 'master' into add-async-insert-mt-setting 2023-05-01 16:13:23 +02:00
Alexander Gololobov
c1c6955374 Deprecate delete-on-destroy.txt, do not create it any more 2023-05-01 14:47:20 +02:00
Duc Canh Le
abe0cfd10f
fix wrong query result when using nullable primary key (#49172) 2023-05-01 12:51:22 +02:00
alesapin
f03e75971a Remove on detach 2023-04-30 19:06:31 +02:00
alesapin
a098298c0f Remove file for empty parts 2023-04-30 19:06:23 +02:00
alesapin
a95a5f6fb6 Fix two bugs and add test 2023-04-30 16:33:23 +02:00
alesapin
ec2c860a39 Merge branch 'fix_zero_copy_not_atomic' of github.com:ClickHouse/ClickHouse into fix_zero_copy_not_atomic 2023-04-30 15:02:27 +02:00
Smita Kulkarni
408db4a25b Updated to store IMergeTreeDataPartInfoForReader instead of DataPart in MergeTreeMarksLoader 2023-04-30 10:56:43 +02:00
Alexey Milovidov
15d0379e2c
Merge pull request #48792 from ClickHouse/fix_some_tests3 
Fix some tests
 2023-04-29 22:04:11 +03:00
Nikita Taranov
f1ff2c0135
Fix oss-fuzz build errors (#49236) 2023-04-29 18:44:50 +02:00
alesapin
70ee02a3eb Add node holder 2023-04-28 17:39:32 +02:00
Alexey Milovidov
911d4e9977
Merge branch 'master' into fix_some_tests3 2023-04-28 17:23:46 +03:00
alesapin
ef49603432 Merge branch 'master' into fix_zero_copy_not_atomic 2023-04-28 14:42:30 +02:00
Nikita Taranov
71a092d201 Merge branch 'master' into optimize_reading2 2023-04-28 12:29:27 +00:00
Raúl Marín
bb4a7f65de
Slight improvements to coordinator logging (#49204) 2023-04-28 13:25:53 +02:00
Nikita Taranov
de03b90570 impl 2023-04-27 17:07:00 +00:00
Alexey Milovidov
3fa19c5ee2
Merge branch 'master' into fix_some_tests3 2023-04-27 15:04:15 +03:00
Alexander Tokmakov
fc9b2fbac8
Fix race on Outdated parts loading (#49223) 2023-04-27 13:52:09 +02:00
Alexey Milovidov
354cf09892
Merge pull request #48549 from cangyin/add_zookeeper_name 
add zookeeper name in system.replicas
 2023-04-27 01:22:22 +03:00
alesapin
a34e465abf Fxi 2023-04-26 19:57:18 +02:00
Alexey Milovidov
ea34df25f4
Merge branch 'master' into add_zookeeper_name 2023-04-26 13:53:57 +03:00
Robert Schulze
bc313184a5
Merge branch 'master' into hlee-s390x-part-hash 2023-04-25 18:56:40 +02:00
Robert Schulze
1c41967906
Merge remote-tracking branch 'origin/master' into hlee-s390x-part-hash 2023-04-25 09:11:00 +00:00
cangyin
623c8db1a1 add column zookeeper_name in system.replicas 2023-04-25 09:19:36 +08:00
Anton Popov
c7db58be66 add MergeTree setting async_insert 2023-04-25 01:11:58 +00:00
Robert Schulze
6ba6e24a5a
Merge pull request #48536 from lingtaolf/feature/support_settings_allow_suspicious_indices 
Reject identical expressions in indices (allow_suspicious_indices)
 2023-04-24 22:27:59 +02:00
Anton Popov
38622d0770 add settings to delay or throw in case of too many mutations 2023-04-24 18:21:49 +00:00
Alexander Tokmakov
19f43ee7c7 Merge branch 'master' into fix_some_tests3 2023-04-24 16:13:27 +02:00
Robert Schulze
744e7c7601
Some fixups 2023-04-24 09:43:36 +00:00
Alexey Milovidov
3ae7bf754b Merge branch 'master' into compress-marks-primary-key-by-default 2023-04-23 22:46:13 +02:00
Nikita Taranov
54c1b73b5d
Revert "Adapt marks count for prefetch read pool" 2023-04-23 15:33:32 +02:00
Kseniia Sumarokova
7f6488cf37
Merge pull request #49051 from nickitat/adapt_for_prefetch_read_pool 
Adapt marks count for prefetch read pool
 2023-04-23 12:12:47 +02:00
Kseniia Sumarokova
be0cb808de
Merge pull request #48973 from kssenii/randomize-fs-read-settings 
Randomize prefetch settings in ci
 2023-04-23 12:04:30 +02:00
凌涛
08e81d0a52
Merge branch 'master' into feature/support_settings_allow_suspicious_indices 2023-04-23 10:08:00 +08:00
Nikita Taranov
d99c227b0d impl 2023-04-22 21:13:23 +00:00
kssenii
e0eb88f9bc Fix clang-tidy 2023-04-22 17:02:17 +02:00
kssenii
475a893be3 Fix, add assertions 2023-04-21 21:52:17 +02:00
robot-clickhouse-ci-1
32ac001448
Merge pull request #48838 from amosbird/issue_48735 
Fix key condition on duplicate primary keys
 2023-04-21 12:46:50 +02:00
kssenii
0ccc3afe74 Fix 2023-04-21 12:26:28 +02:00
凌涛
797d5bbfe7 modify code 2023-04-21 18:08:14 +08:00
凌涛
de18bf72b4 modify code 2023-04-21 17:27:26 +08:00
Smita Kulkarni
aa9635e35b Removed is_part_outdated flag & usage, updated MergeTreeMarksLoader to hold data_part instead of data_part_storage 2023-04-21 11:24:02 +02:00
kssenii
88784568dc Better 2023-04-20 23:24:04 +02:00
kssenii
6edb61015d Fix style check 2023-04-20 19:35:49 +02:00
kssenii
0a34f3bcea Merge remote-tracking branch 'upstream/master' into randomize-fs-read-settings 2023-04-20 19:04:01 +02:00
kssenii
6552ec6afc Small improvement 2023-04-20 18:15:09 +02:00
Smita Kulkarni
780eef674f Merge branch 'master' into Clear_marks_cache_for_outdated_parts 2023-04-20 16:17:26 +02:00
kssenii
59211c490a More correct accounting of limits 2023-04-20 15:43:14 +02:00
alesapin
d8e6c6ab88
Merge pull request #48731 from ClickHouse/limit_mutations_per_entry 
Limit the number of mutations commands per `MUTATE_PART` entry
 2023-04-20 15:05:46 +02:00
kssenii
ec1370b2ed Fix 2023-04-20 14:46:10 +02:00
Antonio Andelic
09156ab2b7 Merge branch 'master' into keeper-check-not-exists 2023-04-20 12:07:16 +00:00
Antonio Andelic
fb16623d48 Add CheckNotExists request to Keeper 2023-04-18 14:45:24 +00:00
Anton Popov
c06b580890
Merge pull request #48844 from CheSema/wait_loading_parts 
fix the race wait loading parts
 2023-04-18 14:58:20 +02:00
凌涛
a975211185 add more tests 2023-04-18 10:26:35 +08:00
Alexander Gololobov
ba5ca15c40
Merge pull request #46835 from ClickHouse/reduce_mem_in_mutation_with_subquery 
Reduce memory consumption by mutations with big subqueries used with IN
 2023-04-17 23:55:08 +02:00
凌涛
d1a14900ea remove unused h 2023-04-17 20:26:31 +08:00
凌涛
7ed4c413e3 add settings allow_suspicious_indices 2023-04-17 20:23:19 +08:00
Amos Bird
97d27c87a6
Fix key condition on duplicate primary keys 
Fix key condition usage when encountering duplicate primary keys. This can happen in projections. This bug was introduced in https://github.com/ClickHouse/ClickHouse/pull/45686.

This PR is also an optimization because we don't need to create unused FieldRefs anymore.
 2023-04-17 19:13:42 +08:00
Sema Checherinda
daa55ae0d2 fix the race wait loading parts 2023-04-17 13:01:28 +02:00
Michael Kolupaev
87be78e6de Better 2023-04-17 04:58:32 +00:00
Alexander Tokmakov
70c1000c22 fix some tests 2023-04-14 19:42:17 +02:00
Dmitry Novik
cf5d9a175a Revert "Merge pull request #48760 from ClickHouse/revert-46089-background-memory-tracker" 
This reverts commit a61ed33223, reversing
changes made to 5f01b8a2b5.
 2023-04-14 16:34:19 +02:00
Alexander Gololobov
7a381e9232 Added enable_sharing_sets_for_mutations setting 2023-04-14 16:12:18 +02:00
Alexander Gololobov
633aaa73aa Use use_index_for_in_with_subqueries_max_values setting to limit sizes of sets used in KeyCondition 2023-04-14 16:12:18 +02:00
Alexander Gololobov
28d0cc974a Cleanups 2023-04-14 16:12:18 +02:00
Alexander Gololobov
adbe87e9d3 Fixes 2023-04-14 16:12:18 +02:00
Alexander Gololobov
0b9579bfb8 Reworked, testing 2023-04-14 16:12:18 +02:00
Alexander Gololobov
02876ed87d Share PreparedSet-s between task of the same mutation to reduce mem usage 2023-04-14 16:12:18 +02:00
Alexander Tokmakov
de9dc2b0b3
Merge branch 'master' into limit_mutations_per_entry 2023-04-14 15:19:08 +03:00
Alexander Tokmakov
af1bf08663
Revert "Add MemoryTracker for the background tasks" 2023-04-13 21:05:02 +03:00
Alexander Tokmakov
8d207e8752 Merge branch 'master' into limit_mutations_per_entry 2023-04-13 18:13:44 +02:00
Alexander Tokmakov
93368b4d55
Merge pull request #48730 from ClickHouse/remove_some_code 
Do not remove broken detached parts on startup
 2023-04-13 16:18:59 +03:00
Dmitry Novik
467ecf45e1
Merge pull request #46089 from ClickHouse/background-memory-tracker 
Add MemoryTracker for the background tasks
 2023-04-13 13:29:27 +02:00
Alexander Tokmakov
d6bff4c5a8 limit the number of mutations commands per MUTATE_PART 2023-04-13 00:20:43 +02:00
Alexander Tokmakov
5d77bb149e do not remove broken detached parts on startup 2023-04-12 23:26:34 +02:00
robot-ch-test-poll2
d1793a9bc5
Merge pull request #48692 from ClickHouse/rs/mergetree-syntax 
Update MergeTree syntax for optional index granularity argument
 2023-04-12 21:35:40 +02:00
alesapin
8067cfcb51
Merge pull request #46946 from ClickHouse/revert-46909-revert-45911-mutations_rename_hang 
Fix several `RENAME COLUMN` bugs
 2023-04-12 19:07:48 +02:00
Dmitry Novik
6c11acccc9 Remove unnecessary code 2023-04-12 14:08:09 +00:00
Robert Schulze
6b441b40dd
Update MergeTree syntax for optional index granularity argument 
This is follow up to #45451.
 2023-04-12 11:16:55 +00:00
alesapin
55c3f1ee29
Merge pull request #48666 from ClickHouse/remove_strange_code_from_mutation_task 
Remove strange code from MutateTask
 2023-04-12 12:38:43 +02:00
SmitaRKulkarni
7ad05c1bc3
Merge branch 'master' into Clear_marks_cache_for_outdated_parts 2023-04-12 09:21:59 +02:00
SmitaRKulkarni
dd64eaed66
Merge pull request #47985 from Algunenano/zk_retry_timeout 
Do not continue retrying to connect to ZK if the query is killed or over limits
 2023-04-12 08:49:47 +02:00
alesapin
f91309d83d Fix drop in compact parts 2023-04-11 21:21:42 +02:00
alesapin
a72ee0b85e Merge remote-tracking branch 'origin/master' into revert-46909-revert-45911-mutations_rename_hang 2023-04-11 20:14:43 +02:00
alesapin
165edd332e Remove strange code from MutateTask 2023-04-11 18:31:53 +02:00
Dmitry Novik
06e6794fc0 Merge remote-tracking branch 'origin/master' into background-memory-tracker 2023-04-11 15:29:35 +00:00
SmitaRKulkarni
a55d344898
Merge branch 'master' into Clear_marks_cache_for_outdated_parts 2023-04-11 17:00:31 +02:00
Anton Popov
5b8db57fe8
Merge branch 'master' into retry-loading-of-parts 2023-04-11 15:51:40 +02:00
SmitaRKulkarni
e2010c0257
Merge branch 'master' into zk_retry_timeout 2023-04-11 14:05:52 +02:00
Sergei Trifonov
ca6e253fa6
Merge pull request #48526 from ClickHouse/lost-part-count 2023-04-11 13:11:42 +02:00
Sema Checherinda
4dd86a406a
Merge pull request #48543 from azat/mv-uniq-thread-group 
Use one ThreadGroup while pushing to materialized views (and some refactoring for ThreadGroup)
 2023-04-11 11:47:46 +02:00
Azat Khuzhin
79b83c4fd2 Remove superfluous includes of logger_userful.h from headers 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-10 17:59:30 +02:00
Raúl Marín
45ad555c39
Merge branch 'master' into zk_retry_timeout 2023-04-10 10:04:16 +02:00
Sergei Trifonov
b319271d59
Remove misleading comment 
It is no longer "significant" after `NOEXCEPT_SCOPE` macro rework #39229
 2023-04-08 22:50:24 +02:00
robot-clickhouse-ci-1
3302f1ff90
Merge pull request #48299 from CurtizJ/do-not-build-unnecessary-sets 
Do not build sets for skip indexes if they are not used
 2023-04-08 21:25:06 +02:00
Alexander Tokmakov
bf20e6bfc6
Merge pull request #48550 from ClickHouse/fix_sync_replica 
Fix some tests
 2023-04-08 14:41:10 +03:00
Alexander Tokmakov
e39ee3f0af suppress a bug in filesystem cache 2023-04-07 22:54:49 +02:00
Alexander Tokmakov
18a2eb355e fix a bug in sync replica 2023-04-07 20:16:25 +02:00
Azat Khuzhin
aacf2a0838 Move ThreadGroupSwitcher to ThreadStatus.h (out from MergeTree code) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-07 15:32:08 +02:00
Azat Khuzhin
5b2b20a0b0 Rename ThreadGroupStatus to ThreadGroup 
There are methods like getThreadGroup() and ThreadGroupSwitcher class,
so seems that this is logical.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-07 15:31:48 +02:00
Azat Khuzhin
011480924a Use forward declaration of ThreadPool 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-07 11:25:35 +02:00
serxa
343a071798 add lost_part_count column to system.replicas 2023-04-06 22:38:58 +00:00
Smita Kulkarni
7b33744618 Added a check to clear and not load marks asynchronously from outdated parts 2023-04-06 20:11:40 +02:00
Sema Checherinda
b8729be97a
Merge pull request #47104 from CheSema/thead-group-for-merge 
merge and mutation make thread group for setting memory trackers right
 2023-04-06 15:46:56 +02:00
Vladimir C
e7a5c96353
Merge pull request #48219 from ClickHouse/vdimir/lock_order_47072 2023-04-06 11:54:33 +02:00
robot-ch-test-poll3
3ad0a6ac18
Merge pull request #48433 from ClickHouse/some-fixes-for-parallel-replicas 
Some fixes for parallel replicas
 2023-04-06 02:00:06 +02:00
Anton Popov
4fb7e0ec3c add retries to loading of data parts 2023-04-05 21:23:57 +00:00
Sema Checherinda
554c928534
Merge branch 'master' into thead-group-for-merge 2023-04-05 21:09:38 +02:00
Sema Checherinda
c1adbb7aa6 transfer counters to the query directly 2023-04-05 19:51:36 +02:00
robot-clickhouse
3d2c279859
Merge pull request #48242 from azat/better-throttling 
Per-query/per-server throttling for remote IO/local IO/BACKUPs
 2023-04-05 19:32:37 +02:00
Nikita Mikhaylov
3f44c6d936 Better 2023-04-05 15:19:45 +00:00
vdimir
01ce3baf0f
Rename Context::getTemporaryVolume to getGlobalTemporaryVolume 2023-04-05 13:42:13 +00:00
Alexander Tokmakov
3b5ed9242c
Merge pull request #48391 from azat/fix-sync-replica-stuck 
Fix possible SYSTEM SYNC REPLICA stuck in case of DROP/REPLACE PARTITION
 2023-04-05 15:20:46 +03:00
Nikita Mikhaylov
7b526d3df9
Always communicate with local replica through loopback interface for parallel replicas (#48246) 2023-04-05 13:32:46 +02:00
Sema Checherinda
f504cd5fed avoid race, do not set thread_group.counters.parent from query 2023-04-05 12:36:55 +02:00
Azat Khuzhin
218b1f9c29 Add ability to throttle BACKUPs on per-server/backup basis 
Server settings:
- backup_read_bandwidth_for_server
- backup_write_bandwidth_for_server

Query settings:
- backup_read_bandwidth
- backup_write_bandwidth

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:48 +02:00
Raúl Marín
0e17b3b147 Merge remote-tracking branch 'blessed/master' into zk_retry_timeout 2023-04-04 18:53:55 +02:00
Nikita Mikhaylov
fa5b2bd4a5
Added Keeper retries for backups operations (#47224) 2023-04-04 18:12:08 +02:00
Azat Khuzhin
353434f8f1 Fix possible SYSTEM SYNC REPLICA stuck in case of DROP/REPLACE PARTITION 
In case of DROP/REPLACE PARTITION the entries from the queue will be
removed without notifying the subscribes.

CI: https://s3.amazonaws.com/clickhouse-test-reports/48242/4a315cd0d17bc12edd934fd25663b0119880f207/stress_test__asan_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-04 17:16:04 +02:00
Yakov Olkhovskiy
35e9e45249
Merge pull request #48062 from Algunenano/unnecessary_alter_checks 
Only check MV on ALTER when necessary
 2023-04-03 17:23:11 -04:00
Sema Checherinda
6d5edc0315 rename fake_query_context to task_context 2023-04-03 22:01:18 +02:00
Anton Popov
207a4c6166
Merge branch 'master' into do-not-build-unnecessary-sets 2023-04-03 14:29:33 +02:00
Raúl Marín
8fdf87982c Merge remote-tracking branch 'blessed/master' into zk_retry_timeout 2023-04-03 10:26:18 +02:00
Alexey Milovidov
ef1e282d4c
Merge branch 'master' into compress-marks-primary-key-by-default 2023-04-03 06:28:48 +03:00
Alexey Milovidov
5ebf66819b
Merge pull request #47617 from ClickHouse/cnf-new-analyzer 
CNF/Constraint optimizer in new analyzer
 2023-04-02 20:09:34 +03:00
Anton Popov
cdf9cf2fd4 do not build sets for indexes if they are not used 2023-03-31 16:06:20 +00:00
Alexander Tokmakov
75ffc8451b fix 2023-03-31 16:09:00 +02:00
Alexander Tokmakov
caf00b5968 Merge branch 'master' into sync_replica_lightweight 2023-03-31 14:52:55 +02:00
Alexey Milovidov
17c2678675
Merge branch 'master' into compress-marks-primary-key-by-default 2023-03-31 15:15:38 +03:00
Azat Khuzhin
f38a7aeabe ThreadPool metrics introspection 
There are lots of thread pools and simple local-vs-global is not enough
already, it is good to know which one in particular uses threads.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-03-29 10:46:59 +02:00
Antonio Andelic
50ed205aa0
Merge pull request #47303 from ClickHouse/avoid-merging-partition 
Skip merged partitions for `min_age_to_force_merge_seconds` merges
 2023-03-29 09:31:00 +02:00
Sema Checherinda
1031e2001b fix build 2023-03-29 07:41:31 +02:00
HarryLeeIBM
47b31c39d1 Fix partition id issue for s390x 2023-03-28 15:31:26 -07:00
Robert Schulze
622d1dfadd
Merge pull request #46252 from ClibMouse/feature/hastokencaseinsensitive-index-utilization 
Implement tokenbf_v1 index utilization for hasTokenCaseInsensitive
 2023-03-28 22:38:21 +02:00
Sema Checherinda
b774310dc9 keep query id for logging when optimize query 2023-03-28 16:44:47 +02:00
Kruglov Pavel
292539b90e
Merge pull request #47981 from kssenii/better-delegate-disks-handle-mutual-path 
Fix case when (optional) path is not added to encrypted disk configuration
 2023-03-28 13:52:50 +02:00
Raúl Marín
bffde5e476 Merge remote-tracking branch 'blessed/master' into unnecessary_alter_checks 2023-03-28 13:17:36 +02:00
Vladimir C
5fa519d043
Merge pull request #48047 from ClickHouse/vdimir/max_number_of_mutatuins_for_replica 2023-03-28 10:50:35 +02:00
Antonio Andelic
0b5fd2140a Merge branch 'master' into cnf-new-analyzer 2023-03-28 07:57:12 +00:00
Antonio Andelic
30e4ee705a Better 2023-03-28 07:54:23 +00:00
Antonio Andelic
a3b6d43212
Merge branch 'master' into avoid-merging-partition 2023-03-28 09:40:36 +02:00
Alexander Tokmakov
b5fe1ada0b more options for sync replica 2023-03-28 01:39:36 +02:00
kssenii
314ee12442 Fix checks 2023-03-27 19:20:11 +02:00
Igor Nikonov
c2c7895b82
Merge pull request #47770 from ClickHouse/stop_wait_for_quorum_retries_on_shutdown 
Stop `wait for quorum` retries on shutdown
 2023-03-27 18:27:14 +02:00
Raúl Marín
d1a6c1991a Only check MV on ALTER when necessary 2023-03-27 17:45:15 +02:00
Sema Checherinda
a6ab33a906 no use query, but storage context 2023-03-27 16:30:32 +02:00
Raúl Marín
83b68caccc Do not continue retrying to connect to ZK if the query is killed or over limits 2023-03-27 16:01:15 +02:00
vdimir
08fa1846b1
Add merge tree setting max_number_of_mutations_for_replica 2023-03-27 11:56:48 +00:00
Antonio Andelic
e4a78a982f Merge branch 'master' into cnf-new-analyzer 2023-03-27 07:43:21 +00:00
Smita Kulkarni
eabcefe452 Fixed issue with reserving vector size 2023-03-26 14:52:08 +02:00
SmitaRKulkarni
f0b724497f
Merge branch 'master' into 47200_Fix_for_index_file_cidx_is_unexpectedly_long 2023-03-26 14:02:56 +02:00
Smita Kulkarni
decdf0c507 Updated to delete added keys if there is an exception during update of metadata cache 2023-03-26 10:35:44 +02:00
Nikolai Kochetov
327674f56d
Merge pull request #46537 from ClickHouse/aggregate-projections-analysis-query-plan 
Projections analysis from query plan
 2023-03-25 13:23:30 +01:00
Sema Checherinda
aeb8766ad5 adjust after rebase 2023-03-24 19:53:16 +01:00
Sema Checherinda
da3e744405 set context from the master thread 2023-03-24 19:53:16 +01:00
Sema Checherinda
0fcf7c0363 std::optional instead shared_ptr 2023-03-24 19:53:16 +01:00
Sema Checherinda
6a6d45e6e7 set up performance_counters for thread group 2023-03-24 19:53:16 +01:00
Sema Checherinda
da4f2bd923 do not attach empty thread group 2023-03-24 19:53:16 +01:00
Sema Checherinda
bc107c70fa merge and mutation make thread group for setting memory trackers right 2023-03-24 19:53:16 +01:00
kssenii
16ebfcc3f8 Better delegate disks handle mutual path 2023-03-24 16:21:19 +01:00
Nikolai Kochetov
a2182f2659
Merge pull request #47874 from amosbird/correct_exact_rows_before_limit 
Correct `exact_rows_before_limit` in some scenarios
 2023-03-24 15:09:42 +01:00
Alexander Tokmakov
4dde73c3eb
Merge pull request #47917 from ClickHouse/fix_terminate_in_check_thread 
Fix terminate in parts check thread
 2023-03-24 04:22:00 +03:00
Nikolai Kochetov
73e98de46d Merge branch 'master' into aggregate-projections-analysis-query-plan 2023-03-23 21:28:36 +01:00
ltrk2
f46970671d Implement tokenbf_v1 index utilization for hasTokenCaseInsensitive 2023-03-23 12:24:13 -07:00
Maksim Kita
24b9e5103f
Merge pull request #47748 from ClickHouse/revert-47721-revert-47622-analyzer-planner-fixes-before-enable-by-default 
Revert "Revert "Analyzer planner fixes before enable by default""
 2023-03-23 22:20:34 +03:00
Alexander Tokmakov
5b263b9ffe fix 2023-03-23 15:36:17 +01:00
Maksim Kita
7da53bf7e8 Fixed tests 2023-03-23 12:34:23 +01:00
robot-ch-test-poll3
a31c0d8eee
Merge pull request #47883 from ClickHouse/drop-column-with-projections 
Fix DROP COLUMN with ReplicatedMergeTree containing projections
 2023-03-22 23:55:16 +01:00
Alexander Tokmakov
e3c798bae5 fix terminate in parts check thread 2023-03-22 23:46:15 +01:00
Maksim Kita
f45dac99c2 Revert "Revert "Analyzer planner fixes before enable by default"" 2023-03-22 17:19:32 +01:00
Nikolai Kochetov
d7d3722b76 Revert analyzer till later. 2023-03-22 15:41:54 +00:00
Amos Bird
02c5d1f364
Correct exact_rows_before_limit in all senarios 2023-03-22 23:26:31 +08:00
Antonio Andelic
b832840cf3 Recalculate always projections on mutations in ReplicatedMergeTree 2023-03-22 11:21:23 +00:00
Alexander Tokmakov
c114361879
Update MergeTreeData.h 2023-03-21 23:15:59 +03:00
Alexander Tokmakov
8a71e42f08
Merge pull request #47769 from HarryLeeIBM/hlee-s390x-partid 
Fix partition ID byte order for s390x
 2023-03-21 23:01:36 +03:00
Alexander Tokmakov
0989dd837a
Merge pull request #47802 from ClickHouse/fix_some_tests 
Fix 993 and two other tests
 2023-03-21 22:20:06 +03:00
Nikolai Kochetov
b432fa1122 Attempt to use new analyzer. 2023-03-21 17:56:58 +00:00
Antonio Andelic
ed3f51d9b3 Merge branch 'master' into avoid-merging-partition 2023-03-21 10:46:03 +00:00
Vladimir C
5add1e1b2e
Merge pull request #47684 from ClickHouse/vdimir/improve_log_queries_cut 2023-03-21 11:17:54 +01:00
Alexander Tokmakov
d5a561a2ef fix 993 and two other tests 2023-03-20 22:51:32 +01:00
Robert Schulze
595796e63d
Merge branch 'master' into rs/more-llvm-16-preparation 2023-03-20 21:16:27 +01:00
HarryLeeIBM
ed1ff02114 Fix partition ID byte order for s390x 2023-03-20 14:19:27 -04:00
vdimir
bb7a8ed862
Apply log_queries_cut_to_length in MergeTreeWhereOptimizer 2023-03-20 15:38:09 +00:00
Igor Nikonov
1073c7f3c9 Revert "Actually stop retries in case of storage shutdown" 
This was unnecessary since, as for exceptions, retries done only on
zkutil::KeeperException

This reverts commit 795a1c84e1.
 2023-03-20 13:54:25 +00:00
Igor Nikonov
795a1c84e1 Actually stop retries in case of storage shutdown 2023-03-20 13:48:04 +00:00
Igor Nikonov
1098055409 Stop wait for quorum retries on shutdown 2023-03-20 13:21:10 +00:00