thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 03:12:43 +00:00
Code Issues Packages Projects Releases Wiki Activity
117,551 Commits 9,326 Branches 1,783 Tags 2.1 GiB
241d711264
Commit Graph

5955 Commits

Author SHA1 Message Date
Robert Schulze
2b74daaa17
Cosmetics: make input switch a bit more idiomatic 2023-06-05 09:29:03 +00:00
Robert Schulze
a3d4ede26c
Cosmetics: Update exception messages 2023-06-05 09:29:01 +00:00
Robert Schulze
7608e08eed
Cosmetics: more constness 2023-06-05 09:28:58 +00:00
Robert Schulze
423f692282
Cosmetics: Remove unnecessary toString() 2023-06-05 09:28:55 +00:00
Robert Schulze
3b77e40902
Cosmetics: Remove dots from exception messages 2023-06-05 09:28:53 +00:00
Robert Schulze
c5ededdc5b
Cosmetics: Switch arguments in MTIConditionAnnoy ctor 2023-06-05 09:28:50 +00:00
Robert Schulze
594572b0de
Cosmetics: AnnoyIndex --> AnnoyIndexWithSerialization 2023-06-05 09:28:47 +00:00
Robert Schulze
dc88d4e742
Cosmetics: Factorize repeated typedefs into a single typedef 2023-06-05 09:28:44 +00:00
Robert Schulze
e373cf682a
Cosmetics: Unwrap Annoy index from nested namespace 2023-06-05 09:28:41 +00:00
Robert Schulze
f8c1f2bd33
Cosmetics: Remove absolute namespace qualification of Annoy library internals 2023-06-05 09:28:38 +00:00
Robert Schulze
a8bf7af291
Cosmetics: Move ErrorCodes to top 2023-06-05 09:28:35 +00:00
Robert Schulze
4631595cf6
Cosmetics: number_of_trees --> trees 2023-06-05 09:28:32 +00:00
Robert Schulze
1018677f46
Cosmetics: Move ctors into cpp file 2023-06-05 09:28:29 +00:00
Robert Schulze
edad92a7f2
Cosmetics: Minor aesthetic fixes 2023-06-05 09:28:26 +00:00
Robert Schulze
3275629230
Cosmetics: Remove parentheses in single statement if/for/while 2023-06-05 09:28:24 +00:00
Robert Schulze
3657ef05ff
Cosmetics: Fix indentation 2023-06-05 09:28:19 +00:00
Alexey Milovidov
3c5bd78856 Fix typo 2023-06-05 06:13:39 +02:00
Alexey Milovidov
881bdbf839
Merge pull request #50153 from amosbird/fix_44504 
Fix invalid index analysis for date related keys
 2023-06-05 05:00:20 +03:00
Alexey Milovidov
5fc8838b04
Update KeyCondition.cpp 2023-06-05 04:58:29 +03:00
Alexey Milovidov
3b8c781b2c
Merge pull request #50448 from ClickHouse/rm_parts_from_zk_paranoid 
Paranoid fix for removing parts from ZooKeeper
 2023-06-05 04:42:15 +03:00
Amos Bird
e24c9267bc
fix 2023-06-04 20:06:27 +08:00
Amos Bird
97bd3f0483
Proper mutation of skip indices and projections 2023-06-04 18:32:08 +08:00
Alexey Milovidov
9e938d7eec
Merge pull request #49287 from nickitat/optimize_reading2 
Optimise hot reading from remote tables
 2023-06-02 23:51:57 +03:00
Alexander Tokmakov
4209ccfc08 fix 2023-06-02 17:42:04 +02:00
Alexander Tokmakov
e148c60d5a
Fixes for MergeTree with readonly disks (#50244) 
* fixes for MergeTree with readonly disks

* Automatic style fix

* Update test.py

* Automatic style fix

* Update test.py

* Update test.py

* Automatic style fix

* Update test.py

---------

Co-authored-by: robot-clickhouse <robot-clickhouse@users.noreply.github.com>
Co-authored-by: alesapin <alesapin@clickhouse.com>
 2023-06-02 16:18:32 +03:00
Alexey Milovidov
31773d7eee
Merge pull request #50432 from azat/fix-min-for-seek 
Fix merge_tree_min_rows_for_seek/merge_tree_min_bytes_for_seek for data skipping indexes
 2023-06-02 04:09:35 +03:00
Alexey Milovidov
60dccff77c
Merge pull request #50177 from amosbird/fix_50094 
Fix broken index analysis when binary operator contains a null constant argument
 2023-06-02 01:29:03 +03:00
Alexander Tokmakov
f62faaedc3 paranoid fix for removing parts from zk 2023-06-01 21:20:39 +02:00
Azat Khuzhin
53eb360ac2 Fix merge_tree_min_rows_for_seek/merge_tree_min_bytes_for_seek for data skipping indexes 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-06-01 15:04:16 +02:00
SmitaRKulkarni
06432929a7
Merge pull request #50339 from ClickHouse/43983_Updated_to_hold_lock_fetch_part 
Improved fetch part by holding directory lock longer
 2023-06-01 13:42:34 +02:00
Nikita Mikhaylov
3543d95980
Parallel replicas always skip unavailable ones (#50293) 2023-05-31 22:10:33 +02:00
SmitaRKulkarni
123cd505ee
Merge branch 'master' into 43983_Updated_to_hold_lock_fetch_part 2023-05-31 21:59:55 +02:00
Alexander Tokmakov
a18b998076
Merge branch 'master' into rmt_better_background_tasks_scheduling 2023-05-31 19:47:29 +03:00
Alexander Tokmakov
82e1d5ae67 Merge branch 'master' into rmt_better_background_tasks_scheduling 2023-05-30 17:40:13 +02:00
Smita Kulkarni
fe944a55dc Updated to hold temporary_directory_lock till part is fetched, renamed and committed 2023-05-30 13:30:47 +02:00
Sema Checherinda
2d2b411c26 fix build 2023-05-30 12:26:20 +02:00
Sema Checherinda
0b4ea3e2e1 remove reminder comments 2023-05-30 12:26:20 +02:00
Sema Checherinda
a5dcd8dabb do not call finalize in d-tor MergedBlockOutputStream::Finalizer 2023-05-30 12:26:20 +02:00
Sema Checherinda
fe3939287b add test, fix .gin_sid files 2023-05-30 12:26:20 +02:00
Nikita Taranov
be62c8faba fix build 2023-05-29 15:27:39 +02:00
Nikita Taranov
a3652f1727 Merge branch 'master' into optimize_reading2 2023-05-29 15:26:18 +02:00
alesapin
3a3cee586a
Merge pull request #50257 from CurtizJ/useless-refactoring 
Yet another refactoring
 2023-05-28 21:42:16 +02:00
Nikita Taranov
42a9072888 Merge branch 'master' into optimize_reading2 2023-05-28 18:04:44 +02:00
Sergei Trifonov
0d1f2e297b
Unify priorities: lower value means higher priority (#50205) 2023-05-26 15:55:30 +02:00
Anton Popov
612173e734 refactoring near alter conversions 2023-05-25 22:54:54 +00:00
Kseniia Sumarokova
f1a3c9cfd5
Merge pull request #50109 from kssenii/abstract-async-prefetched-buffer 
Make async prefetched buffer work with arbitrary impl
 2023-05-25 15:06:44 +02:00
Nikita Mikhaylov
1c3b6738f4
Fixes for parallel replicas (#50195) 2023-05-25 14:41:04 +02:00
Alexander Tokmakov
d8f936987e Merge branch 'master' into rmt_better_background_tasks_scheduling 2023-05-25 12:36:41 +02:00
何李夫
e4c8c4cecf
Add zookeeper name in endpoint id (#49780) 
* Add zookeeper name in endpoint id

When we migrate a replicated table from one zookeeper cluster to
another (the reason why we migration is that zookeeper's load is
too high), we will create a new table with the same zpath, but it
will fail and the old table will be in trouble.

Here is some infomation:
1.old table:
  CREATE TABLE a1 (`id` UInt64)
  ENGINE = ReplicatedMergeTree('/clickhouse/tables/default/a1/{shard}', '{replica}')
  ORDER BY (id);
2.new table:
  CREATE TABLE a2 (`id` UInt64)
  ENGINE = ReplicatedMergeTree('aux1:/clickhouse/tables/default/a1/{shard}', '{replica}')
  ORDER BY (id);
3.error info:
  <Error> executeQuery: Code: 220. DB::Exception: Duplicate interserver IO endpoint:
          DataPartsExchange:/clickhouse/tables/default/a1/01/replicas/02.
          (DUPLICATE_INTERSERVER_IO_ENDPOINT)
  <Error> InterserverIOHTTPHandler: Code: 221. DB::Exception: No interserver IO endpoint
          named DataPartsExchange:/clickhouse/tables/default/a1/01/replicas/02.
          (NO_SUCH_INTERSERVER_IO_ENDPOINT)

* Revert "Add zookeeper name in endpoint id"

This reverts commit 9deb75b249619b7abdd38e3949ca8b3a76c9df8e.

* Add zookeeper name in endpoint id

When we migrate a replicated table from one zookeeper cluster to
another (the reason why we migration is that zookeeper's load is
too high), we will create a new table with the same zpath, but it
will fail and the old table will be in trouble.

* Fix incompatible with a new setting

* add a test, fix other issues

* Update 02442_auxiliary_zookeeper_endpoint_id.sql

* Update 02735_system_zookeeper_connection.reference

* Update 02735_system_zookeeper_connection.sql

* Update run.sh

* Remove the 'no-fasttest' tag

* Update 02442_auxiliary_zookeeper_endpoint_id.sql

---------

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2023-05-25 12:50:14 +03:00
alesapin
7c0c49c9d2
Merge pull request #50154 from hanfei1991/hanfei/fix-modify-order-by 
do not allow modify order by when there are no order by cols
 2023-05-24 15:01:38 +02:00
Alexander Tokmakov
486153d581
Update MergeTreeData.cpp 2023-05-24 13:33:28 +03:00
Amos Bird
8bbfdcc56c
Fix index analysis with binary operator null 2023-05-24 15:47:38 +08:00
Amos Bird
b11aa42db9
Fix tests 2023-05-24 14:27:49 +08:00
Alexey Milovidov
3e1267c839
Merge pull request #50152 from ClickHouse/tavplubix-patch-6 
Follow-up to #49889
 2023-05-24 01:05:24 +03:00
Han Fei
037c5f8a06
Merge branch 'master' into hanfei/fix-modify-order-by 2023-05-23 20:48:27 +02:00
Igor Nikonov
2c01104c3f Clarification comment on retries controller behavior 2023-05-23 17:30:22 +00:00
Han Fei
584c05d8b8 fix modify order by when there was no order by cols 2023-05-23 18:54:36 +02:00
Raúl Marín
db4b3d19ae
Clearer coordinator log (#50101) 2023-05-23 17:30:27 +02:00
Alexander Tokmakov
64ee8ebb12
Update MutateTask.cpp 2023-05-23 18:11:08 +03:00
Amos Bird
b82ff979d0
Fix invalid index analysis for date related keys 2023-05-23 23:10:34 +08:00
Alexander Tokmakov
dbf08b25fb better scheduling of merge selecting task 2023-05-23 01:25:17 +02:00
Alexander Tokmakov
ec6f07aa78 Merge branch 'less_zookeeper_requests' into rmt_better_background_tasks_scheduling 2023-05-22 23:51:36 +02:00
kssenii
295fe3b228 Merge remote-tracking branch 'upstream/master' into abstract-async-prefetched-buffer 2023-05-22 20:01:38 +02:00
kssenii
c4d862a16f Make async reader work with any impl 2023-05-22 19:54:04 +02:00
Alexander Tokmakov
b8305503d8 more flexible cleanup thread scheduling 2023-05-22 19:07:18 +02:00
Alexander Tokmakov
821b64b420 apply review suggestions 2023-05-22 15:18:29 +02:00
Alexander Tokmakov
487e510103 Merge branch 'master' into less_zookeeper_requests 2023-05-22 13:59:26 +02:00
Alexander Tokmakov
c89f92e1f6
Merge pull request #50052 from amosbird/fix_49913 
Fix reporting broken projection parts
 2023-05-22 14:20:06 +03:00
Antonio Andelic
93cd47fd5f
Merge pull request #50026 from ClickHouse/fix-deadlock-attach-thread 
Avoid deadlock when starting table in attach thread of `ReplicatedMergeTree`
 2023-05-22 09:22:33 +02:00
Amos Bird
0a3d986e42
Fix reporting projection broken part 2023-05-21 20:58:58 +08:00
Nikita Taranov
c93836b962 fix 2023-05-19 22:27:53 +00:00
Antonio Andelic
4af8187464 Activate restarting thread in both cases 2023-05-19 15:06:02 +00:00
Antonio Andelic
3107070e76 Avoid deadlock when starting table in attach thread 2023-05-19 12:48:19 +00:00
alesapin
e741450b88
Merge branch 'master' into fix_another_zero_copy_bug 2023-05-19 12:40:48 +02:00
alesapin
e5b001abda
Merge branch 'master' into fix_some_tests4 2023-05-19 12:34:03 +02:00
Michael Kolupaev
e84f0895e7 Support hardlinking parts transactionally 2023-05-18 21:05:56 -07:00
Nikita Taranov
971cc092d4
Update src/Storages/MergeTree/MergeTreePrefetchedReadPool.cpp 2023-05-18 15:16:47 +02:00
Kseniia Sumarokova
1c04085e8f
Update MergeTreeWriteAheadLog.h 2023-05-17 18:15:51 +02:00
Nikita Taranov
0dd67bacf2
Merge branch 'master' into optimize_reading2 2023-05-17 15:06:41 +02:00
Alexander Tokmakov
36c31e1d79
Improve concurrent parts removal with zero copy replication (#49630) 
* improve concurrent parts removal

* fix

* fix
 2023-05-17 14:07:34 +03:00
Vitaly Baranov
6c8a923c9d
Merge branch 'master' into write-encrypted-to-backup 2023-05-17 12:37:05 +02:00
Alexander Tokmakov
242a3fc520 Merge branch 'master' into less_zookeeper_requests 2023-05-16 18:24:11 +02:00
Alexander Tokmakov
0da82945ac fix 2023-05-16 18:18:48 +02:00
alesapin
50a536bba8 Remove unused code 2023-05-16 15:26:24 +02:00
Alexander Tokmakov
b6716a8f0f Merge branch 'master' into fix_some_tests4 2023-05-16 14:46:27 +02:00
Vitaly Baranov
943707963f Add backup setting "decrypt_files_from_encrypted_disks" 2023-05-16 14:27:27 +02:00
Vitaly Baranov
5198997fd8 Remove ReadSettings from backup entries. 2023-05-16 14:27:27 +02:00
Vitaly Baranov
cc50fcc60a Remove the 'temporary_file_' argument from BackupEntryFromImmutableFile's constructor. 2023-05-16 14:25:37 +02:00
alesapin
93bd09ddd6
Merge branch 'master' into fix_another_zero_copy_bug 2023-05-16 12:24:52 +02:00
alesapin
0b4ab70dd9
Merge pull request #49891 from hanfei1991/hanfei/chassert-1 
use chassert in MergeTreeDeduplicationLog to have better log info
 2023-05-16 11:50:11 +02:00
Sema Checherinda
03c51208d1
Merge pull request #44869 from CheSema/multi_part_upload 
rework WriteBufferFromS3, add tests, add abortion
 2023-05-16 10:52:01 +02:00
Alexander Tokmakov
65bc702b0b fix 2023-05-15 20:02:30 +02:00
Han Fei
4137a5e058 use chassert in MergeTreeDeduplicationLog to have better log info 2023-05-15 18:51:16 +02:00
Alexander Tokmakov
05ae7b2c2d fix some tests 2023-05-15 18:28:12 +02:00
Alexey Milovidov
b4b5b4dcc5 Amend 2023-05-14 02:52:23 +02:00
Alexey Milovidov
0ca36d4f89 Merge branch 'master' into clang-17 2023-05-14 01:57:40 +02:00
Alexey Milovidov
a2d1cc1333
Merge pull request #49844 from MikhailBurdukov/in_with_tuple 
TYPE_MISMATCH exception for in operator with  single column tuples.
 2023-05-13 04:21:12 +03:00
Alexey Milovidov
5a44dc26e7 Fixes for clang-17 2023-05-13 02:57:31 +02:00
MikhailBurdukov
8c7b63f1e7 Fix 2023-05-12 16:42:01 +00:00
Robert Schulze
b9c185af44
Merge pull request #49678 from azat/build/llvm-16 
Switch to LLVM/clang 16 (16.0.3)
 2023-05-12 13:47:36 +02:00
Sema Checherinda
a1ee7d52e1
Merge pull request #49432 from CheSema/lost-blobs 
all s3-blobs removed when merge aborted, remove part from failed fetch without unlock keper
 2023-05-12 13:19:27 +02:00
Sema Checherinda
5706e05757 adding more comments 2023-05-12 13:00:56 +02:00
Nikita Taranov
536c5ac261
Merge branch 'master' into optimize_reading2 2023-05-12 12:59:32 +02:00
Sema Checherinda
66e3712dca
Merge branch 'master' into lost-blobs 2023-05-11 08:23:25 +02:00
Sema Checherinda
9724688608 rename field, do not fetch PreActive 2023-05-11 08:22:37 +02:00
Azat Khuzhin
d56e1423ef
Merge branch 'master' into build/llvm-16 2023-05-10 21:23:26 +02:00
Sema Checherinda
7fbf87be17 rework WriteBufferFromS3, squashed 2023-05-10 18:31:47 +00:00
Alexander Gololobov
049ef9dda5
Merge branch 'master' into fix_lwd_and_object 2023-05-10 19:54:26 +02:00
Nikita Taranov
ec63a6ee7c
Merge branch 'master' into optimize_reading2 2023-05-10 17:24:37 +02:00
Alexey Milovidov
4a4eb5b171
Merge pull request #49660 from CurtizJ/fix-sparse-columns-reload 
Fix reading from sparse columns after restart
 2023-05-10 18:08:22 +03:00
Alexey Milovidov
33a58aacea
Merge pull request #49719 from ClickHouse/make-projections-non-experimental 
Make projections production-ready
 2023-05-10 18:07:02 +03:00
Azat Khuzhin
a0cc5b7e47 Merge remote-tracking branch 'u/master' into build/llvm-16 2023-05-10 16:28:36 +02:00
Alexander Gololobov
e6f4e5cd10 Ignore LWD column in checkPartDynamicColumns 2023-05-10 16:10:04 +02:00
Han Fei
d17aa828b3
Merge pull request #49361 from hanfei1991/hanfei/fiu 
support named fault injection and add a simple test
 2023-05-10 15:51:51 +02:00
Roman Vasin
632c0eeefc
Add support of temporary tables in FROM part of ATTACH|REPLACE PARTITION FROM (#49436) 
* Add temp tables after FROM

* Resolve works fine with temp and non-temp tables

* Add usage of temporary table in  00626_replace_partition_from_table test

* Update documentation

* Add tests for ReplicatedMergeTree

* Move tests into 02731_replace_partition_from_temporary_table
 2023-05-10 15:25:12 +03:00
Alexey Milovidov
8a6e07f0ea Make projections production-ready 2023-05-10 03:35:13 +02:00
Anton Popov
996fcfe120 fix column ttl with sparse columns 2023-05-09 21:37:43 +00:00
Alexander Tokmakov
6f53ee289d fix another issue, add a test 2023-05-09 23:26:56 +02:00
Azat Khuzhin
d03ae2abfa Fix modernize-loop-convert clang-tidy warning 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-09 21:19:37 +02:00
Alexey Milovidov
a8bdb20fc4
Merge pull request #48787 from ClickHouse/background-memory-tracker 
Add MemoryTracker for the background tasks [Resubmit]
 2023-05-09 07:58:36 +03:00
Alexey Milovidov
9a73a04fe3
Merge pull request #42587 from ClickHouse/compress-marks-primary-key-by-default 
Compress marks and primary key by default
 2023-05-09 06:43:10 +03:00
Han Fei
64b67b9a37 Merge branch 'master' into hanfei/fiu 2023-05-09 01:14:30 +02:00
Alexander Tokmakov
aa829c4ddc better partitions hint in mutations finalization 2023-05-09 00:23:21 +02:00
Han Fei
82ce47ce56 address comments 2023-05-08 23:02:36 +02:00
Anton Popov
0612f5d782 fix sparse columns after reload 2023-05-08 11:50:21 +00:00
Alexey Milovidov
3cf1da7bf3
Merge branch 'master' into deprecate-in-memory-parts 2023-05-08 01:38:00 +03:00
Alexander Tokmakov
91489243c9 provide better partitions hint for merge selecting task 2023-05-08 00:22:07 +02:00
Alexander Tokmakov
29bbde85a6 split selectPartsToMerge into smaller functions 2023-05-07 22:06:25 +02:00
alesapin
33e691ad51 Better name 2023-05-07 14:26:21 +02:00
alesapin
a2ed457219 Better concurrent parts removal with zero copy 2023-05-07 14:02:24 +02:00
Alexey Milovidov
3b6edbacab Update tests 2023-05-07 03:45:17 +02:00
Alexey Milovidov
de3e66385c Merge branch 'master' into compress-marks-primary-key-by-default 2023-05-07 02:40:25 +02:00
Sema Checherinda
08bfc8f58d inserted parts has exlusive blobs 2023-05-07 01:51:43 +02:00
Sema Checherinda
f302c1986a use flag has_exclusive_blobs to track the part's originate 2023-05-06 03:07:29 +02:00
Alexey Milovidov
75a2589a42
Merge branch 'master' into deprecate-in-memory-parts 2023-05-06 00:49:59 +03:00
Robert Schulze
3dfc0bd265
Merge pull request #49413 from azat/build/headers 
Slightly reduce inter-header dependencies
 2023-05-05 23:37:58 +02:00
Nikolay Degterinsky
7a727bbbe3
Merge pull request #49520 from IvanTakarlikov-st/it-grammar-fixes 
Fix some grammar mistakes in documentation, code and tests
 2023-05-05 15:45:58 +02:00
Sema Checherinda
9a0bc891e4 fix can_remove_callback for Temporary parts 2023-05-04 19:39:16 +02:00
Ivan Takarlikov
8873856ce5 Fix some grammar mistakes in documentation, code and tests 2023-05-04 13:35:18 -03:00
Anton Popov
e61d7baa54 Merge remote-tracking branch 'upstream/master' into HEAD 2023-05-04 14:03:53 +00:00
Alexey Milovidov
261907c143 Merge branch 'master' into deprecate-in-memory-parts 2023-05-04 01:07:38 +02:00
alesapin
610e63bfd2 Fix build 2023-05-03 23:54:24 +02:00
wangxiaobo
2924837c7e
Fix Attaching a table with non-existent ZK path does not increase the ReadonlyReplica metric (#48954) 
* Attaching a table with non-existent ZK path does not increase the ReadonlyReplica metric bug fix.

* add integration test.

* add integration test.

* Update test.py

* format code

* format code
 2023-05-03 22:41:20 +03:00
alesapin
913b63edc9 Fix another zero copy bug 2023-05-03 21:28:33 +02:00
Anton Popov
c9705d6bf1
Merging #49066 (Better error handling during loading of parts) (#49430) 
* catch filesystem error when loading parts

* better exception handling during part loading

---------

Co-authored-by: cangyin <excangyin@gmail.com>
Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
 2023-05-03 19:10:06 +02:00
Nikita Mikhaylov
954e3b724c
Speedup outdated parts loading (#49317) 2023-05-03 18:56:45 +02:00
Anton Popov
7896d30737
Merge pull request #49122 from CurtizJ/add-async-insert-mt-setting 
Add `MergeTree` setting `async_insert`
 2023-05-03 16:01:13 +02:00
Nikita Taranov
30d216f863 fix 2023-05-03 13:49:46 +00:00
alesapin
6f3f202f7c
Merge branch 'master' into fix_zero_copy_not_atomic 2023-05-03 12:53:35 +02:00
Alexey Milovidov
f449df85b6 Deprecate in-memory parts 2023-05-03 00:31:09 +02:00