thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 19:32:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
119,460 Commits 9,319 Branches 1,783 Tags 2.1 GiB
d8dd480cd7
Commit Graph

6083 Commits

Author SHA1 Message Date
Nikita Taranov
a3652f1727 Merge branch 'master' into optimize_reading2 2023-05-29 15:26:18 +02:00
alesapin
3a3cee586a
Merge pull request #50257 from CurtizJ/useless-refactoring 
Yet another refactoring
 2023-05-28 21:42:16 +02:00
Nikita Taranov
42a9072888 Merge branch 'master' into optimize_reading2 2023-05-28 18:04:44 +02:00
Nikolai Kochetov
6ed7a3b73f Fixing more tests. 2023-05-26 19:25:33 +00:00
Sergei Trifonov
0d1f2e297b
Unify priorities: lower value means higher priority (#50205) 2023-05-26 15:55:30 +02:00
Anton Popov
612173e734 refactoring near alter conversions 2023-05-25 22:54:54 +00:00
Nikolai Kochetov
b5b261b22c Merge branch 'master' into refactor-subqueries-for-in 2023-05-25 21:22:06 +02:00
Nikolai Kochetov
30ff5113d9 Fixing more tests. 2023-05-25 13:33:52 +00:00
Kseniia Sumarokova
f1a3c9cfd5
Merge pull request #50109 from kssenii/abstract-async-prefetched-buffer 
Make async prefetched buffer work with arbitrary impl
 2023-05-25 15:06:44 +02:00
Nikita Mikhaylov
1c3b6738f4
Fixes for parallel replicas (#50195) 2023-05-25 14:41:04 +02:00
Alexander Tokmakov
d8f936987e Merge branch 'master' into rmt_better_background_tasks_scheduling 2023-05-25 12:36:41 +02:00
何李夫
e4c8c4cecf
Add zookeeper name in endpoint id (#49780) 
* Add zookeeper name in endpoint id

When we migrate a replicated table from one zookeeper cluster to
another (the reason why we migration is that zookeeper's load is
too high), we will create a new table with the same zpath, but it
will fail and the old table will be in trouble.

Here is some infomation:
1.old table:
  CREATE TABLE a1 (`id` UInt64)
  ENGINE = ReplicatedMergeTree('/clickhouse/tables/default/a1/{shard}', '{replica}')
  ORDER BY (id);
2.new table:
  CREATE TABLE a2 (`id` UInt64)
  ENGINE = ReplicatedMergeTree('aux1:/clickhouse/tables/default/a1/{shard}', '{replica}')
  ORDER BY (id);
3.error info:
  <Error> executeQuery: Code: 220. DB::Exception: Duplicate interserver IO endpoint:
          DataPartsExchange:/clickhouse/tables/default/a1/01/replicas/02.
          (DUPLICATE_INTERSERVER_IO_ENDPOINT)
  <Error> InterserverIOHTTPHandler: Code: 221. DB::Exception: No interserver IO endpoint
          named DataPartsExchange:/clickhouse/tables/default/a1/01/replicas/02.
          (NO_SUCH_INTERSERVER_IO_ENDPOINT)

* Revert "Add zookeeper name in endpoint id"

This reverts commit 9deb75b249619b7abdd38e3949ca8b3a76c9df8e.

* Add zookeeper name in endpoint id

When we migrate a replicated table from one zookeeper cluster to
another (the reason why we migration is that zookeeper's load is
too high), we will create a new table with the same zpath, but it
will fail and the old table will be in trouble.

* Fix incompatible with a new setting

* add a test, fix other issues

* Update 02442_auxiliary_zookeeper_endpoint_id.sql

* Update 02735_system_zookeeper_connection.reference

* Update 02735_system_zookeeper_connection.sql

* Update run.sh

* Remove the 'no-fasttest' tag

* Update 02442_auxiliary_zookeeper_endpoint_id.sql

---------

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2023-05-25 12:50:14 +03:00
alesapin
7c0c49c9d2
Merge pull request #50154 from hanfei1991/hanfei/fix-modify-order-by 
do not allow modify order by when there are no order by cols
 2023-05-24 15:01:38 +02:00
Alexander Tokmakov
486153d581
Update MergeTreeData.cpp 2023-05-24 13:33:28 +03:00
Amos Bird
8bbfdcc56c
Fix index analysis with binary operator null 2023-05-24 15:47:38 +08:00
Amos Bird
b11aa42db9
Fix tests 2023-05-24 14:27:49 +08:00
Alexey Milovidov
3e1267c839
Merge pull request #50152 from ClickHouse/tavplubix-patch-6 
Follow-up to #49889
 2023-05-24 01:05:24 +03:00
Han Fei
037c5f8a06
Merge branch 'master' into hanfei/fix-modify-order-by 2023-05-23 20:48:27 +02:00
Nikolai Kochetov
6de52e9fce Fixing some tests. 2023-05-23 18:36:02 +00:00
Igor Nikonov
2c01104c3f Clarification comment on retries controller behavior 2023-05-23 17:30:22 +00:00
Han Fei
584c05d8b8 fix modify order by when there was no order by cols 2023-05-23 18:54:36 +02:00
Raúl Marín
db4b3d19ae
Clearer coordinator log (#50101) 2023-05-23 17:30:27 +02:00
Alexander Tokmakov
64ee8ebb12
Update MutateTask.cpp 2023-05-23 18:11:08 +03:00
Amos Bird
b82ff979d0
Fix invalid index analysis for date related keys 2023-05-23 23:10:34 +08:00
Alexander Tokmakov
dbf08b25fb better scheduling of merge selecting task 2023-05-23 01:25:17 +02:00
Alexander Tokmakov
ec6f07aa78 Merge branch 'less_zookeeper_requests' into rmt_better_background_tasks_scheduling 2023-05-22 23:51:36 +02:00
kssenii
295fe3b228 Merge remote-tracking branch 'upstream/master' into abstract-async-prefetched-buffer 2023-05-22 20:01:38 +02:00
kssenii
c4d862a16f Make async reader work with any impl 2023-05-22 19:54:04 +02:00
Alexander Tokmakov
b8305503d8 more flexible cleanup thread scheduling 2023-05-22 19:07:18 +02:00
Alexander Tokmakov
821b64b420 apply review suggestions 2023-05-22 15:18:29 +02:00
Alexander Tokmakov
487e510103 Merge branch 'master' into less_zookeeper_requests 2023-05-22 13:59:26 +02:00
Alexander Tokmakov
c89f92e1f6
Merge pull request #50052 from amosbird/fix_49913 
Fix reporting broken projection parts
 2023-05-22 14:20:06 +03:00
Nikolai Kochetov
ff6400f91a Merge branch 'master' into refactor-subqueries-for-in 2023-05-22 13:07:51 +02:00
Antonio Andelic
93cd47fd5f
Merge pull request #50026 from ClickHouse/fix-deadlock-attach-thread 
Avoid deadlock when starting table in attach thread of `ReplicatedMergeTree`
 2023-05-22 09:22:33 +02:00
Amos Bird
0a3d986e42
Fix reporting projection broken part 2023-05-21 20:58:58 +08:00
Nikita Taranov
c93836b962 fix 2023-05-19 22:27:53 +00:00
zvonand
7dccb51bc0 fix conflict and update 2023-05-19 23:40:51 +02:00
Antonio Andelic
4af8187464 Activate restarting thread in both cases 2023-05-19 15:06:02 +00:00
Antonio Andelic
3107070e76 Avoid deadlock when starting table in attach thread 2023-05-19 12:48:19 +00:00
alesapin
e741450b88
Merge branch 'master' into fix_another_zero_copy_bug 2023-05-19 12:40:48 +02:00
alesapin
e5b001abda
Merge branch 'master' into fix_some_tests4 2023-05-19 12:34:03 +02:00
Michael Kolupaev
e84f0895e7 Support hardlinking parts transactionally 2023-05-18 21:05:56 -07:00
Nikita Taranov
971cc092d4
Update src/Storages/MergeTree/MergeTreePrefetchedReadPool.cpp 2023-05-18 15:16:47 +02:00
Kseniia Sumarokova
1c04085e8f
Update MergeTreeWriteAheadLog.h 2023-05-17 18:15:51 +02:00
Nikita Taranov
0dd67bacf2
Merge branch 'master' into optimize_reading2 2023-05-17 15:06:41 +02:00
Alexander Tokmakov
36c31e1d79
Improve concurrent parts removal with zero copy replication (#49630) 
* improve concurrent parts removal

* fix

* fix
 2023-05-17 14:07:34 +03:00
Vitaly Baranov
6c8a923c9d
Merge branch 'master' into write-encrypted-to-backup 2023-05-17 12:37:05 +02:00
Alexander Tokmakov
242a3fc520 Merge branch 'master' into less_zookeeper_requests 2023-05-16 18:24:11 +02:00
Alexander Tokmakov
0da82945ac fix 2023-05-16 18:18:48 +02:00
alesapin
50a536bba8 Remove unused code 2023-05-16 15:26:24 +02:00
Alexander Tokmakov
b6716a8f0f Merge branch 'master' into fix_some_tests4 2023-05-16 14:46:27 +02:00
Vitaly Baranov
943707963f Add backup setting "decrypt_files_from_encrypted_disks" 2023-05-16 14:27:27 +02:00
Vitaly Baranov
5198997fd8 Remove ReadSettings from backup entries. 2023-05-16 14:27:27 +02:00
Vitaly Baranov
cc50fcc60a Remove the 'temporary_file_' argument from BackupEntryFromImmutableFile's constructor. 2023-05-16 14:25:37 +02:00
alesapin
93bd09ddd6
Merge branch 'master' into fix_another_zero_copy_bug 2023-05-16 12:24:52 +02:00
alesapin
0b4ab70dd9
Merge pull request #49891 from hanfei1991/hanfei/chassert-1 
use chassert in MergeTreeDeduplicationLog to have better log info
 2023-05-16 11:50:11 +02:00
Sema Checherinda
03c51208d1
Merge pull request #44869 from CheSema/multi_part_upload 
rework WriteBufferFromS3, add tests, add abortion
 2023-05-16 10:52:01 +02:00
Alexander Tokmakov
65bc702b0b fix 2023-05-15 20:02:30 +02:00
Han Fei
4137a5e058 use chassert in MergeTreeDeduplicationLog to have better log info 2023-05-15 18:51:16 +02:00
Alexander Tokmakov
05ae7b2c2d fix some tests 2023-05-15 18:28:12 +02:00
Alexey Milovidov
b4b5b4dcc5 Amend 2023-05-14 02:52:23 +02:00
Alexey Milovidov
0ca36d4f89 Merge branch 'master' into clang-17 2023-05-14 01:57:40 +02:00
Alexey Milovidov
a2d1cc1333
Merge pull request #49844 from MikhailBurdukov/in_with_tuple 
TYPE_MISMATCH exception for in operator with  single column tuples.
 2023-05-13 04:21:12 +03:00
Alexey Milovidov
5a44dc26e7 Fixes for clang-17 2023-05-13 02:57:31 +02:00
MikhailBurdukov
8c7b63f1e7 Fix 2023-05-12 16:42:01 +00:00
Robert Schulze
b9c185af44
Merge pull request #49678 from azat/build/llvm-16 
Switch to LLVM/clang 16 (16.0.3)
 2023-05-12 13:47:36 +02:00
Sema Checherinda
a1ee7d52e1
Merge pull request #49432 from CheSema/lost-blobs 
all s3-blobs removed when merge aborted, remove part from failed fetch without unlock keper
 2023-05-12 13:19:27 +02:00
Sema Checherinda
5706e05757 adding more comments 2023-05-12 13:00:56 +02:00
Nikita Taranov
536c5ac261
Merge branch 'master' into optimize_reading2 2023-05-12 12:59:32 +02:00
Sema Checherinda
66e3712dca
Merge branch 'master' into lost-blobs 2023-05-11 08:23:25 +02:00
Sema Checherinda
9724688608 rename field, do not fetch PreActive 2023-05-11 08:22:37 +02:00
zvonand
ecd0b4cc06 update settings from master 2023-05-11 00:37:58 +02:00
Azat Khuzhin
d56e1423ef
Merge branch 'master' into build/llvm-16 2023-05-10 21:23:26 +02:00
Sema Checherinda
7fbf87be17 rework WriteBufferFromS3, squashed 2023-05-10 18:31:47 +00:00
Alexander Gololobov
049ef9dda5
Merge branch 'master' into fix_lwd_and_object 2023-05-10 19:54:26 +02:00
Nikita Taranov
ec63a6ee7c
Merge branch 'master' into optimize_reading2 2023-05-10 17:24:37 +02:00
Alexey Milovidov
4a4eb5b171
Merge pull request #49660 from CurtizJ/fix-sparse-columns-reload 
Fix reading from sparse columns after restart
 2023-05-10 18:08:22 +03:00
Alexey Milovidov
33a58aacea
Merge pull request #49719 from ClickHouse/make-projections-non-experimental 
Make projections production-ready
 2023-05-10 18:07:02 +03:00
Azat Khuzhin
a0cc5b7e47 Merge remote-tracking branch 'u/master' into build/llvm-16 2023-05-10 16:28:36 +02:00
Alexander Gololobov
e6f4e5cd10 Ignore LWD column in checkPartDynamicColumns 2023-05-10 16:10:04 +02:00
Han Fei
d17aa828b3
Merge pull request #49361 from hanfei1991/hanfei/fiu 
support named fault injection and add a simple test
 2023-05-10 15:51:51 +02:00
zvonand
7762dde325 Merge branch 'master' of github.com:ClickHouse/ClickHouse into zvonand-implicit-tz 2023-05-10 14:49:17 +02:00
Roman Vasin
632c0eeefc
Add support of temporary tables in FROM part of ATTACH|REPLACE PARTITION FROM (#49436) 
* Add temp tables after FROM

* Resolve works fine with temp and non-temp tables

* Add usage of temporary table in  00626_replace_partition_from_table test

* Update documentation

* Add tests for ReplicatedMergeTree

* Move tests into 02731_replace_partition_from_temporary_table
 2023-05-10 15:25:12 +03:00
Alexey Milovidov
8a6e07f0ea Make projections production-ready 2023-05-10 03:35:13 +02:00
Andrey Zvonov
9237b904be
Merge branch 'master' into zvonand-implicit-tz 2023-05-10 01:25:33 +02:00
Anton Popov
996fcfe120 fix column ttl with sparse columns 2023-05-09 21:37:43 +00:00
Alexander Tokmakov
6f53ee289d fix another issue, add a test 2023-05-09 23:26:56 +02:00
Azat Khuzhin
d03ae2abfa Fix modernize-loop-convert clang-tidy warning 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-09 21:19:37 +02:00
Alexey Milovidov
a8bdb20fc4
Merge pull request #48787 from ClickHouse/background-memory-tracker 
Add MemoryTracker for the background tasks [Resubmit]
 2023-05-09 07:58:36 +03:00
Alexey Milovidov
9a73a04fe3
Merge pull request #42587 from ClickHouse/compress-marks-primary-key-by-default 
Compress marks and primary key by default
 2023-05-09 06:43:10 +03:00
Han Fei
64b67b9a37 Merge branch 'master' into hanfei/fiu 2023-05-09 01:14:30 +02:00
Alexander Tokmakov
aa829c4ddc better partitions hint in mutations finalization 2023-05-09 00:23:21 +02:00
Han Fei
82ce47ce56 address comments 2023-05-08 23:02:36 +02:00
Anton Popov
0612f5d782 fix sparse columns after reload 2023-05-08 11:50:21 +00:00
Alexey Milovidov
3cf1da7bf3
Merge branch 'master' into deprecate-in-memory-parts 2023-05-08 01:38:00 +03:00
Alexander Tokmakov
91489243c9 provide better partitions hint for merge selecting task 2023-05-08 00:22:07 +02:00
Alexander Tokmakov
29bbde85a6 split selectPartsToMerge into smaller functions 2023-05-07 22:06:25 +02:00
alesapin
33e691ad51 Better name 2023-05-07 14:26:21 +02:00
alesapin
a2ed457219 Better concurrent parts removal with zero copy 2023-05-07 14:02:24 +02:00
Alexey Milovidov
3b6edbacab Update tests 2023-05-07 03:45:17 +02:00
Alexey Milovidov
de3e66385c Merge branch 'master' into compress-marks-primary-key-by-default 2023-05-07 02:40:25 +02:00
Sema Checherinda
08bfc8f58d inserted parts has exlusive blobs 2023-05-07 01:51:43 +02:00
Sema Checherinda
f302c1986a use flag has_exclusive_blobs to track the part's originate 2023-05-06 03:07:29 +02:00
Alexey Milovidov
75a2589a42
Merge branch 'master' into deprecate-in-memory-parts 2023-05-06 00:49:59 +03:00
Robert Schulze
3dfc0bd265
Merge pull request #49413 from azat/build/headers 
Slightly reduce inter-header dependencies
 2023-05-05 23:37:58 +02:00
Nikolai Kochetov
618486815b Merge branch 'master' into refactor-subqueries-for-in 2023-05-05 20:39:09 +02:00
Nikolai Kochetov
54ef6769f1 Refactor PreparedSets [4] 2023-05-05 18:30:08 +00:00
zvonand
8338d54c34 Merge branch 'master' of github.com:ClickHouse/ClickHouse into zvonand-implicit-tz 2023-05-05 15:50:26 +02:00
Nikolay Degterinsky
7a727bbbe3
Merge pull request #49520 from IvanTakarlikov-st/it-grammar-fixes 
Fix some grammar mistakes in documentation, code and tests
 2023-05-05 15:45:58 +02:00
Nikolai Kochetov
f598a39ea2 Refactor PreparedSets [3] 2023-05-04 17:54:08 +00:00
Sema Checherinda
9a0bc891e4 fix can_remove_callback for Temporary parts 2023-05-04 19:39:16 +02:00
Ivan Takarlikov
8873856ce5 Fix some grammar mistakes in documentation, code and tests 2023-05-04 13:35:18 -03:00
Anton Popov
e61d7baa54 Merge remote-tracking branch 'upstream/master' into HEAD 2023-05-04 14:03:53 +00:00
Alexey Milovidov
261907c143 Merge branch 'master' into deprecate-in-memory-parts 2023-05-04 01:07:38 +02:00
alesapin
610e63bfd2 Fix build 2023-05-03 23:54:24 +02:00
wangxiaobo
2924837c7e
Fix Attaching a table with non-existent ZK path does not increase the ReadonlyReplica metric (#48954) 
* Attaching a table with non-existent ZK path does not increase the ReadonlyReplica metric bug fix.

* add integration test.

* add integration test.

* Update test.py

* format code

* format code
 2023-05-03 22:41:20 +03:00
alesapin
913b63edc9 Fix another zero copy bug 2023-05-03 21:28:33 +02:00
Anton Popov
c9705d6bf1
Merging #49066 (Better error handling during loading of parts) (#49430) 
* catch filesystem error when loading parts

* better exception handling during part loading

---------

Co-authored-by: cangyin <excangyin@gmail.com>
Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
 2023-05-03 19:10:06 +02:00
Nikita Mikhaylov
954e3b724c
Speedup outdated parts loading (#49317) 2023-05-03 18:56:45 +02:00
Anton Popov
7896d30737
Merge pull request #49122 from CurtizJ/add-async-insert-mt-setting 
Add `MergeTree` setting `async_insert`
 2023-05-03 16:01:13 +02:00
Nikita Taranov
30d216f863 fix 2023-05-03 13:49:46 +00:00
alesapin
6f3f202f7c
Merge branch 'master' into fix_zero_copy_not_atomic 2023-05-03 12:53:35 +02:00
Alexey Milovidov
f449df85b6 Deprecate in-memory parts 2023-05-03 00:31:09 +02:00
Azat Khuzhin
2b21b83979 Reduce inter-header dependencies 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-02 21:15:18 +02:00
Nikita Taranov
67bd861bc3 Merge branch 'master' into optimize_reading2 2023-05-02 14:23:56 +00:00
SmitaRKulkarni
997210c782
Merge pull request #48515 from ClickHouse/Clear_marks_cache_for_outdated_parts 
MergeTreeMarksLoader holds DataPart instead of DataPartStorage
 2023-05-02 15:11:14 +02:00
alesapin
24a744db4c Merge branch 'master' into fix_zero_copy_not_atomic 2023-05-02 14:57:55 +02:00
Nikita Mikhaylov
aa4c5fe958
Enhancements for background merges (#49313) 2023-05-02 13:43:59 +02:00
Anton Popov
bcdd2307fb add comments 2023-05-02 11:30:50 +00:00
Antonio Andelic
f695040a19
Merge pull request #48897 from ClickHouse/keeper-check-not-exists 
Add `CheckNotExists` request to Keeper
 2023-05-02 13:14:22 +02:00
Han Fei
d608ba41e7 try to fix 2023-05-02 00:05:36 +02:00
alesapin
0f6a4b973e
Merge branch 'master' into ttl_on_insert 2023-05-01 21:02:04 +02:00
alesapin
a3f9a8b87b Fix bug in removal of existing part directory 2023-05-01 20:32:01 +02:00
Nikita Taranov
979b68a488 fix 2023-05-01 18:24:30 +00:00
Han Fei
4d18a4f8d3
Merge branch 'master' into hanfei/fiu 2023-05-01 17:51:41 +02:00
Han Fei
b701293d46 support named fault injection and add a simple test 2023-05-01 17:44:37 +02:00
alesapin
66847386df
Merge pull request #49146 from ClickHouse/don_t_write_version_file_on_start 
Don't add metadata_version file if it doesn't exist
 2023-05-01 17:04:47 +02:00
Anton Popov
36d53e0d6c
Merge branch 'master' into add-async-insert-mt-setting 2023-05-01 16:13:23 +02:00
Alexander Gololobov
c1c6955374 Deprecate delete-on-destroy.txt, do not create it any more 2023-05-01 14:47:20 +02:00
Duc Canh Le
abe0cfd10f
fix wrong query result when using nullable primary key (#49172) 2023-05-01 12:51:22 +02:00
alesapin
f03e75971a Remove on detach 2023-04-30 19:06:31 +02:00
alesapin
a098298c0f Remove file for empty parts 2023-04-30 19:06:23 +02:00
alesapin
a95a5f6fb6 Fix two bugs and add test 2023-04-30 16:33:23 +02:00
alesapin
ec2c860a39 Merge branch 'fix_zero_copy_not_atomic' of github.com:ClickHouse/ClickHouse into fix_zero_copy_not_atomic 2023-04-30 15:02:27 +02:00
Smita Kulkarni
408db4a25b Updated to store IMergeTreeDataPartInfoForReader instead of DataPart in MergeTreeMarksLoader 2023-04-30 10:56:43 +02:00
Alexey Milovidov
15d0379e2c
Merge pull request #48792 from ClickHouse/fix_some_tests3 
Fix some tests
 2023-04-29 22:04:11 +03:00
Alexey Milovidov
491c26fb0a Slight improvement in Disks interface 2023-04-29 18:55:19 +02:00
Nikita Taranov
f1ff2c0135
Fix oss-fuzz build errors (#49236) 2023-04-29 18:44:50 +02:00
alesapin
70ee02a3eb Add node holder 2023-04-28 17:39:32 +02:00
Alexey Milovidov
911d4e9977
Merge branch 'master' into fix_some_tests3 2023-04-28 17:23:46 +03:00
alesapin
ef49603432 Merge branch 'master' into fix_zero_copy_not_atomic 2023-04-28 14:42:30 +02:00
Nikita Taranov
71a092d201 Merge branch 'master' into optimize_reading2 2023-04-28 12:29:27 +00:00
Raúl Marín
bb4a7f65de
Slight improvements to coordinator logging (#49204) 2023-04-28 13:25:53 +02:00
Nikita Taranov
de03b90570 impl 2023-04-27 17:07:00 +00:00
Alexey Milovidov
3fa19c5ee2
Merge branch 'master' into fix_some_tests3 2023-04-27 15:04:15 +03:00
Alexander Tokmakov
fc9b2fbac8
Fix race on Outdated parts loading (#49223) 2023-04-27 13:52:09 +02:00
Alexey Milovidov
354cf09892
Merge pull request #48549 from cangyin/add_zookeeper_name 
add zookeeper name in system.replicas
 2023-04-27 01:22:22 +03:00
alesapin
a34e465abf Fxi 2023-04-26 19:57:18 +02:00
Alexey Milovidov
ea34df25f4
Merge branch 'master' into add_zookeeper_name 2023-04-26 13:53:57 +03:00
Robert Schulze
bc313184a5
Merge branch 'master' into hlee-s390x-part-hash 2023-04-25 18:56:40 +02:00
Robert Schulze
1c41967906
Merge remote-tracking branch 'origin/master' into hlee-s390x-part-hash 2023-04-25 09:11:00 +00:00
cangyin
623c8db1a1 add column zookeeper_name in system.replicas 2023-04-25 09:19:36 +08:00
Anton Popov
c7db58be66 add MergeTree setting async_insert 2023-04-25 01:11:58 +00:00
Robert Schulze
6ba6e24a5a
Merge pull request #48536 from lingtaolf/feature/support_settings_allow_suspicious_indices 
Reject identical expressions in indices (allow_suspicious_indices)
 2023-04-24 22:27:59 +02:00
Anton Popov
38622d0770 add settings to delay or throw in case of too many mutations 2023-04-24 18:21:49 +00:00
Alexander Tokmakov
19f43ee7c7 Merge branch 'master' into fix_some_tests3 2023-04-24 16:13:27 +02:00
Robert Schulze
744e7c7601
Some fixups 2023-04-24 09:43:36 +00:00
Alexey Milovidov
3ae7bf754b Merge branch 'master' into compress-marks-primary-key-by-default 2023-04-23 22:46:13 +02:00
Nikita Taranov
54c1b73b5d
Revert "Adapt marks count for prefetch read pool" 2023-04-23 15:33:32 +02:00
Kseniia Sumarokova
7f6488cf37
Merge pull request #49051 from nickitat/adapt_for_prefetch_read_pool 
Adapt marks count for prefetch read pool
 2023-04-23 12:12:47 +02:00
Kseniia Sumarokova
be0cb808de
Merge pull request #48973 from kssenii/randomize-fs-read-settings 
Randomize prefetch settings in ci
 2023-04-23 12:04:30 +02:00
凌涛
08e81d0a52
Merge branch 'master' into feature/support_settings_allow_suspicious_indices 2023-04-23 10:08:00 +08:00
Nikita Taranov
d99c227b0d impl 2023-04-22 21:13:23 +00:00
kssenii
e0eb88f9bc Fix clang-tidy 2023-04-22 17:02:17 +02:00
kssenii
475a893be3 Fix, add assertions 2023-04-21 21:52:17 +02:00
robot-clickhouse-ci-1
32ac001448
Merge pull request #48838 from amosbird/issue_48735 
Fix key condition on duplicate primary keys
 2023-04-21 12:46:50 +02:00
kssenii
0ccc3afe74 Fix 2023-04-21 12:26:28 +02:00
凌涛
797d5bbfe7 modify code 2023-04-21 18:08:14 +08:00
凌涛
de18bf72b4 modify code 2023-04-21 17:27:26 +08:00
Smita Kulkarni
aa9635e35b Removed is_part_outdated flag & usage, updated MergeTreeMarksLoader to hold data_part instead of data_part_storage 2023-04-21 11:24:02 +02:00
kssenii
88784568dc Better 2023-04-20 23:24:04 +02:00
Andrey Zvonov
984e8fde41
Merge branch 'master' into zvonand-implicit-tz 2023-04-20 23:13:17 +02:00
kssenii
6edb61015d Fix style check 2023-04-20 19:35:49 +02:00
kssenii
0a34f3bcea Merge remote-tracking branch 'upstream/master' into randomize-fs-read-settings 2023-04-20 19:04:01 +02:00
kssenii
6552ec6afc Small improvement 2023-04-20 18:15:09 +02:00
Smita Kulkarni
780eef674f Merge branch 'master' into Clear_marks_cache_for_outdated_parts 2023-04-20 16:17:26 +02:00
kssenii
59211c490a More correct accounting of limits 2023-04-20 15:43:14 +02:00
alesapin
d8e6c6ab88
Merge pull request #48731 from ClickHouse/limit_mutations_per_entry 
Limit the number of mutations commands per `MUTATE_PART` entry
 2023-04-20 15:05:46 +02:00
kssenii
ec1370b2ed Fix 2023-04-20 14:46:10 +02:00
Antonio Andelic
09156ab2b7 Merge branch 'master' into keeper-check-not-exists 2023-04-20 12:07:16 +00:00
Antonio Andelic
fb16623d48 Add CheckNotExists request to Keeper 2023-04-18 14:45:24 +00:00
Anton Popov
c06b580890
Merge pull request #48844 from CheSema/wait_loading_parts 
fix the race wait loading parts
 2023-04-18 14:58:20 +02:00
凌涛
a975211185 add more tests 2023-04-18 10:26:35 +08:00
Alexander Gololobov
ba5ca15c40
Merge pull request #46835 from ClickHouse/reduce_mem_in_mutation_with_subquery 
Reduce memory consumption by mutations with big subqueries used with IN
 2023-04-17 23:55:08 +02:00
凌涛
d1a14900ea remove unused h 2023-04-17 20:26:31 +08:00
凌涛
7ed4c413e3 add settings allow_suspicious_indices 2023-04-17 20:23:19 +08:00
Amos Bird
97d27c87a6
Fix key condition on duplicate primary keys 
Fix key condition usage when encountering duplicate primary keys. This can happen in projections. This bug was introduced in https://github.com/ClickHouse/ClickHouse/pull/45686.

This PR is also an optimization because we don't need to create unused FieldRefs anymore.
 2023-04-17 19:13:42 +08:00
Sema Checherinda
daa55ae0d2 fix the race wait loading parts 2023-04-17 13:01:28 +02:00
Michael Kolupaev
87be78e6de Better 2023-04-17 04:58:32 +00:00
Alexander Tokmakov
70c1000c22 fix some tests 2023-04-14 19:42:17 +02:00
Dmitry Novik
cf5d9a175a Revert "Merge pull request #48760 from ClickHouse/revert-46089-background-memory-tracker" 
This reverts commit a61ed33223, reversing
changes made to 5f01b8a2b5.
 2023-04-14 16:34:19 +02:00
Alexander Gololobov
7a381e9232 Added enable_sharing_sets_for_mutations setting 2023-04-14 16:12:18 +02:00
Alexander Gololobov
633aaa73aa Use use_index_for_in_with_subqueries_max_values setting to limit sizes of sets used in KeyCondition 2023-04-14 16:12:18 +02:00
Alexander Gololobov
28d0cc974a Cleanups 2023-04-14 16:12:18 +02:00
Alexander Gololobov
adbe87e9d3 Fixes 2023-04-14 16:12:18 +02:00
Alexander Gololobov
0b9579bfb8 Reworked, testing 2023-04-14 16:12:18 +02:00
Alexander Gololobov
02876ed87d Share PreparedSet-s between task of the same mutation to reduce mem usage 2023-04-14 16:12:18 +02:00
Alexander Tokmakov
de9dc2b0b3
Merge branch 'master' into limit_mutations_per_entry 2023-04-14 15:19:08 +03:00
zvonand
089306820f Merge branch 'master' of github.com:ClickHouse/ClickHouse into zvonand-implicit-tz 2023-04-14 13:21:49 +02:00
Alexander Tokmakov
af1bf08663
Revert "Add MemoryTracker for the background tasks" 2023-04-13 21:05:02 +03:00
Alexander Tokmakov
8d207e8752 Merge branch 'master' into limit_mutations_per_entry 2023-04-13 18:13:44 +02:00
Alexander Tokmakov
93368b4d55
Merge pull request #48730 from ClickHouse/remove_some_code 
Do not remove broken detached parts on startup
 2023-04-13 16:18:59 +03:00
Dmitry Novik
467ecf45e1
Merge pull request #46089 from ClickHouse/background-memory-tracker 
Add MemoryTracker for the background tasks
 2023-04-13 13:29:27 +02:00
Alexander Tokmakov
d6bff4c5a8 limit the number of mutations commands per MUTATE_PART 2023-04-13 00:20:43 +02:00
Alexander Tokmakov
5d77bb149e do not remove broken detached parts on startup 2023-04-12 23:26:34 +02:00
robot-ch-test-poll2
d1793a9bc5
Merge pull request #48692 from ClickHouse/rs/mergetree-syntax 
Update MergeTree syntax for optional index granularity argument
 2023-04-12 21:35:40 +02:00
alesapin
8067cfcb51
Merge pull request #46946 from ClickHouse/revert-46909-revert-45911-mutations_rename_hang 
Fix several `RENAME COLUMN` bugs
 2023-04-12 19:07:48 +02:00
Dmitry Novik
6c11acccc9 Remove unnecessary code 2023-04-12 14:08:09 +00:00
Robert Schulze
6b441b40dd
Update MergeTree syntax for optional index granularity argument 
This is follow up to #45451.
 2023-04-12 11:16:55 +00:00
zvonand
2a9f28b73b resolve 2023-04-12 12:54:39 +02:00
zvonand
a9499eed79 moved getting server TZ DateLUT to separate place, upd tests and fix 2023-04-12 12:47:05 +02:00
alesapin
55c3f1ee29
Merge pull request #48666 from ClickHouse/remove_strange_code_from_mutation_task 
Remove strange code from MutateTask
 2023-04-12 12:38:43 +02:00
SmitaRKulkarni
7ad05c1bc3
Merge branch 'master' into Clear_marks_cache_for_outdated_parts 2023-04-12 09:21:59 +02:00
SmitaRKulkarni
dd64eaed66
Merge pull request #47985 from Algunenano/zk_retry_timeout 
Do not continue retrying to connect to ZK if the query is killed or over limits
 2023-04-12 08:49:47 +02:00
alesapin
f91309d83d Fix drop in compact parts 2023-04-11 21:21:42 +02:00
alesapin
a72ee0b85e Merge remote-tracking branch 'origin/master' into revert-46909-revert-45911-mutations_rename_hang 2023-04-11 20:14:43 +02:00
alesapin
165edd332e Remove strange code from MutateTask 2023-04-11 18:31:53 +02:00
Dmitry Novik
06e6794fc0 Merge remote-tracking branch 'origin/master' into background-memory-tracker 2023-04-11 15:29:35 +00:00
SmitaRKulkarni
a55d344898
Merge branch 'master' into Clear_marks_cache_for_outdated_parts 2023-04-11 17:00:31 +02:00
Anton Popov
5b8db57fe8
Merge branch 'master' into retry-loading-of-parts 2023-04-11 15:51:40 +02:00
SmitaRKulkarni
e2010c0257
Merge branch 'master' into zk_retry_timeout 2023-04-11 14:05:52 +02:00
Sergei Trifonov
ca6e253fa6
Merge pull request #48526 from ClickHouse/lost-part-count 2023-04-11 13:11:42 +02:00
Sema Checherinda
4dd86a406a
Merge pull request #48543 from azat/mv-uniq-thread-group 
Use one ThreadGroup while pushing to materialized views (and some refactoring for ThreadGroup)
 2023-04-11 11:47:46 +02:00
Azat Khuzhin
79b83c4fd2 Remove superfluous includes of logger_userful.h from headers 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-10 17:59:30 +02:00
Raúl Marín
45ad555c39
Merge branch 'master' into zk_retry_timeout 2023-04-10 10:04:16 +02:00
Sergei Trifonov
b319271d59
Remove misleading comment 
It is no longer "significant" after `NOEXCEPT_SCOPE` macro rework #39229
 2023-04-08 22:50:24 +02:00
robot-clickhouse-ci-1
3302f1ff90
Merge pull request #48299 from CurtizJ/do-not-build-unnecessary-sets 
Do not build sets for skip indexes if they are not used
 2023-04-08 21:25:06 +02:00
Alexander Tokmakov
bf20e6bfc6
Merge pull request #48550 from ClickHouse/fix_sync_replica 
Fix some tests
 2023-04-08 14:41:10 +03:00
Alexander Tokmakov
e39ee3f0af suppress a bug in filesystem cache 2023-04-07 22:54:49 +02:00
Alexander Tokmakov
18a2eb355e fix a bug in sync replica 2023-04-07 20:16:25 +02:00
Azat Khuzhin
aacf2a0838 Move ThreadGroupSwitcher to ThreadStatus.h (out from MergeTree code) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-07 15:32:08 +02:00
Azat Khuzhin
5b2b20a0b0 Rename ThreadGroupStatus to ThreadGroup 
There are methods like getThreadGroup() and ThreadGroupSwitcher class,
so seems that this is logical.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-07 15:31:48 +02:00
Azat Khuzhin
011480924a Use forward declaration of ThreadPool 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-07 11:25:35 +02:00
serxa
343a071798 add lost_part_count column to system.replicas 2023-04-06 22:38:58 +00:00
Smita Kulkarni
7b33744618 Added a check to clear and not load marks asynchronously from outdated parts 2023-04-06 20:11:40 +02:00
Sema Checherinda
b8729be97a
Merge pull request #47104 from CheSema/thead-group-for-merge 
merge and mutation make thread group for setting memory trackers right
 2023-04-06 15:46:56 +02:00
Vladimir C
e7a5c96353
Merge pull request #48219 from ClickHouse/vdimir/lock_order_47072 2023-04-06 11:54:33 +02:00
robot-ch-test-poll3
3ad0a6ac18
Merge pull request #48433 from ClickHouse/some-fixes-for-parallel-replicas 
Some fixes for parallel replicas
 2023-04-06 02:00:06 +02:00
Anton Popov
4fb7e0ec3c add retries to loading of data parts 2023-04-05 21:23:57 +00:00
Sema Checherinda
554c928534
Merge branch 'master' into thead-group-for-merge 2023-04-05 21:09:38 +02:00
Sema Checherinda
c1adbb7aa6 transfer counters to the query directly 2023-04-05 19:51:36 +02:00
robot-clickhouse
3d2c279859
Merge pull request #48242 from azat/better-throttling 
Per-query/per-server throttling for remote IO/local IO/BACKUPs
 2023-04-05 19:32:37 +02:00
Nikita Mikhaylov
3f44c6d936 Better 2023-04-05 15:19:45 +00:00
vdimir
01ce3baf0f
Rename Context::getTemporaryVolume to getGlobalTemporaryVolume 2023-04-05 13:42:13 +00:00
Alexander Tokmakov
3b5ed9242c
Merge pull request #48391 from azat/fix-sync-replica-stuck 
Fix possible SYSTEM SYNC REPLICA stuck in case of DROP/REPLACE PARTITION
 2023-04-05 15:20:46 +03:00
Nikita Mikhaylov
7b526d3df9
Always communicate with local replica through loopback interface for parallel replicas (#48246) 2023-04-05 13:32:46 +02:00
Sema Checherinda
f504cd5fed avoid race, do not set thread_group.counters.parent from query 2023-04-05 12:36:55 +02:00
Azat Khuzhin
218b1f9c29 Add ability to throttle BACKUPs on per-server/backup basis 
Server settings:
- backup_read_bandwidth_for_server
- backup_write_bandwidth_for_server

Query settings:
- backup_read_bandwidth
- backup_write_bandwidth

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:48 +02:00
Raúl Marín
0e17b3b147 Merge remote-tracking branch 'blessed/master' into zk_retry_timeout 2023-04-04 18:53:55 +02:00
Nikita Mikhaylov
fa5b2bd4a5
Added Keeper retries for backups operations (#47224) 2023-04-04 18:12:08 +02:00
Azat Khuzhin
353434f8f1 Fix possible SYSTEM SYNC REPLICA stuck in case of DROP/REPLACE PARTITION 
In case of DROP/REPLACE PARTITION the entries from the queue will be
removed without notifying the subscribes.

CI: https://s3.amazonaws.com/clickhouse-test-reports/48242/4a315cd0d17bc12edd934fd25663b0119880f207/stress_test__asan_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-04 17:16:04 +02:00
Yakov Olkhovskiy
35e9e45249
Merge pull request #48062 from Algunenano/unnecessary_alter_checks 
Only check MV on ALTER when necessary
 2023-04-03 17:23:11 -04:00
Sema Checherinda
6d5edc0315 rename fake_query_context to task_context 2023-04-03 22:01:18 +02:00
Anton Popov
207a4c6166
Merge branch 'master' into do-not-build-unnecessary-sets 2023-04-03 14:29:33 +02:00
Raúl Marín
8fdf87982c Merge remote-tracking branch 'blessed/master' into zk_retry_timeout 2023-04-03 10:26:18 +02:00
Alexey Milovidov
ef1e282d4c
Merge branch 'master' into compress-marks-primary-key-by-default 2023-04-03 06:28:48 +03:00
Alexey Milovidov
5ebf66819b
Merge pull request #47617 from ClickHouse/cnf-new-analyzer 
CNF/Constraint optimizer in new analyzer
 2023-04-02 20:09:34 +03:00
Anton Popov
cdf9cf2fd4 do not build sets for indexes if they are not used 2023-03-31 16:06:20 +00:00
Alexander Tokmakov
75ffc8451b fix 2023-03-31 16:09:00 +02:00
Alexander Tokmakov
caf00b5968 Merge branch 'master' into sync_replica_lightweight 2023-03-31 14:52:55 +02:00
Alexey Milovidov
17c2678675
Merge branch 'master' into compress-marks-primary-key-by-default 2023-03-31 15:15:38 +03:00
Azat Khuzhin
f38a7aeabe ThreadPool metrics introspection 
There are lots of thread pools and simple local-vs-global is not enough
already, it is good to know which one in particular uses threads.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-03-29 10:46:59 +02:00
Antonio Andelic
50ed205aa0
Merge pull request #47303 from ClickHouse/avoid-merging-partition 
Skip merged partitions for `min_age_to_force_merge_seconds` merges
 2023-03-29 09:31:00 +02:00
Sema Checherinda
1031e2001b fix build 2023-03-29 07:41:31 +02:00
HarryLeeIBM
47b31c39d1 Fix partition id issue for s390x 2023-03-28 15:31:26 -07:00
Robert Schulze
622d1dfadd
Merge pull request #46252 from ClibMouse/feature/hastokencaseinsensitive-index-utilization 
Implement tokenbf_v1 index utilization for hasTokenCaseInsensitive
 2023-03-28 22:38:21 +02:00
Sema Checherinda
b774310dc9 keep query id for logging when optimize query 2023-03-28 16:44:47 +02:00
Kruglov Pavel
292539b90e
Merge pull request #47981 from kssenii/better-delegate-disks-handle-mutual-path 
Fix case when (optional) path is not added to encrypted disk configuration
 2023-03-28 13:52:50 +02:00
Raúl Marín
bffde5e476 Merge remote-tracking branch 'blessed/master' into unnecessary_alter_checks 2023-03-28 13:17:36 +02:00
Vladimir C
5fa519d043
Merge pull request #48047 from ClickHouse/vdimir/max_number_of_mutatuins_for_replica 2023-03-28 10:50:35 +02:00
Antonio Andelic
0b5fd2140a Merge branch 'master' into cnf-new-analyzer 2023-03-28 07:57:12 +00:00
Antonio Andelic
30e4ee705a Better 2023-03-28 07:54:23 +00:00
Antonio Andelic
a3b6d43212
Merge branch 'master' into avoid-merging-partition 2023-03-28 09:40:36 +02:00
Alexander Tokmakov
b5fe1ada0b more options for sync replica 2023-03-28 01:39:36 +02:00
kssenii
314ee12442 Fix checks 2023-03-27 19:20:11 +02:00
Igor Nikonov
c2c7895b82
Merge pull request #47770 from ClickHouse/stop_wait_for_quorum_retries_on_shutdown 
Stop `wait for quorum` retries on shutdown
 2023-03-27 18:27:14 +02:00
Raúl Marín
d1a6c1991a Only check MV on ALTER when necessary 2023-03-27 17:45:15 +02:00
Sema Checherinda
a6ab33a906 no use query, but storage context 2023-03-27 16:30:32 +02:00
Raúl Marín
83b68caccc Do not continue retrying to connect to ZK if the query is killed or over limits 2023-03-27 16:01:15 +02:00
vdimir
08fa1846b1
Add merge tree setting max_number_of_mutations_for_replica 2023-03-27 11:56:48 +00:00
Antonio Andelic
e4a78a982f Merge branch 'master' into cnf-new-analyzer 2023-03-27 07:43:21 +00:00
Smita Kulkarni
eabcefe452 Fixed issue with reserving vector size 2023-03-26 14:52:08 +02:00
SmitaRKulkarni
f0b724497f
Merge branch 'master' into 47200_Fix_for_index_file_cidx_is_unexpectedly_long 2023-03-26 14:02:56 +02:00
Smita Kulkarni
decdf0c507 Updated to delete added keys if there is an exception during update of metadata cache 2023-03-26 10:35:44 +02:00
Nikolai Kochetov
327674f56d
Merge pull request #46537 from ClickHouse/aggregate-projections-analysis-query-plan 
Projections analysis from query plan
 2023-03-25 13:23:30 +01:00
Sema Checherinda
aeb8766ad5 adjust after rebase 2023-03-24 19:53:16 +01:00
Sema Checherinda
da3e744405 set context from the master thread 2023-03-24 19:53:16 +01:00
Sema Checherinda
0fcf7c0363 std::optional instead shared_ptr 2023-03-24 19:53:16 +01:00
Sema Checherinda
6a6d45e6e7 set up performance_counters for thread group 2023-03-24 19:53:16 +01:00
Sema Checherinda
da4f2bd923 do not attach empty thread group 2023-03-24 19:53:16 +01:00