thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-14 18:32:29 +00:00
Code Issues Packages Projects Releases Wiki Activity
116,288 Commits 9,328 Branches 1,783 Tags 2.1 GiB
881bdbf839
Commit Graph

12863 Commits

Author SHA1 Message Date
Nikita Mikhaylov
1c3b6738f4
Fixes for parallel replicas (#50195) 2023-05-25 14:41:04 +02:00
Sema Checherinda
3329a8428d
Merge pull request #49779 from helifu/master3 
Add 'partitions' field for system.query_log
 2023-05-25 12:51:40 +02:00
何李夫
e4c8c4cecf
Add zookeeper name in endpoint id (#49780) 
* Add zookeeper name in endpoint id

When we migrate a replicated table from one zookeeper cluster to
another (the reason why we migration is that zookeeper's load is
too high), we will create a new table with the same zpath, but it
will fail and the old table will be in trouble.

Here is some infomation:
1.old table:
  CREATE TABLE a1 (`id` UInt64)
  ENGINE = ReplicatedMergeTree('/clickhouse/tables/default/a1/{shard}', '{replica}')
  ORDER BY (id);
2.new table:
  CREATE TABLE a2 (`id` UInt64)
  ENGINE = ReplicatedMergeTree('aux1:/clickhouse/tables/default/a1/{shard}', '{replica}')
  ORDER BY (id);
3.error info:
  <Error> executeQuery: Code: 220. DB::Exception: Duplicate interserver IO endpoint:
          DataPartsExchange:/clickhouse/tables/default/a1/01/replicas/02.
          (DUPLICATE_INTERSERVER_IO_ENDPOINT)
  <Error> InterserverIOHTTPHandler: Code: 221. DB::Exception: No interserver IO endpoint
          named DataPartsExchange:/clickhouse/tables/default/a1/01/replicas/02.
          (NO_SUCH_INTERSERVER_IO_ENDPOINT)

* Revert "Add zookeeper name in endpoint id"

This reverts commit 9deb75b249619b7abdd38e3949ca8b3a76c9df8e.

* Add zookeeper name in endpoint id

When we migrate a replicated table from one zookeeper cluster to
another (the reason why we migration is that zookeeper's load is
too high), we will create a new table with the same zpath, but it
will fail and the old table will be in trouble.

* Fix incompatible with a new setting

* add a test, fix other issues

* Update 02442_auxiliary_zookeeper_endpoint_id.sql

* Update 02735_system_zookeeper_connection.reference

* Update 02735_system_zookeeper_connection.sql

* Update run.sh

* Remove the 'no-fasttest' tag

* Update 02442_auxiliary_zookeeper_endpoint_id.sql

---------

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2023-05-25 12:50:14 +03:00
avogar
bc527c7588 Don't send head request for all keys in Iceberg schema inference 2023-05-24 17:07:31 +00:00
alesapin
7c0c49c9d2
Merge pull request #50154 from hanfei1991/hanfei/fix-modify-order-by 
do not allow modify order by when there are no order by cols
 2023-05-24 15:01:38 +02:00
helifu
2255b0287a Add 'partitions' field for system.query_log 2023-05-24 20:42:31 +08:00
Alexander Tokmakov
ffdd916694
Merge pull request #50180 from ClickHouse/tavplubix-patch-6 
Update an exception message
 2023-05-24 15:01:50 +03:00
Kruglov Pavel
9545100c9e
Merge pull request #45427 from attack204/urlCluster 
Add urlCluster table function and refactor all *Cluster table functions
 2023-05-24 13:32:56 +02:00
Alexander Tokmakov
486153d581
Update MergeTreeData.cpp 2023-05-24 13:33:28 +03:00
Amos Bird
8bbfdcc56c
Fix index analysis with binary operator null 2023-05-24 15:47:38 +08:00
Amos Bird
b11aa42db9
Fix tests 2023-05-24 14:27:49 +08:00
Alexey Milovidov
3e1267c839
Merge pull request #50152 from ClickHouse/tavplubix-patch-6 
Follow-up to #49889
 2023-05-24 01:05:24 +03:00
Han Fei
037c5f8a06
Merge branch 'master' into hanfei/fix-modify-order-by 2023-05-23 20:48:27 +02:00
Igor Nikonov
2c01104c3f Clarification comment on retries controller behavior 2023-05-23 17:30:22 +00:00
Han Fei
584c05d8b8 fix modify order by when there was no order by cols 2023-05-23 18:54:36 +02:00
Raúl Marín
db4b3d19ae
Clearer coordinator log (#50101) 2023-05-23 17:30:27 +02:00
Alexander Tokmakov
64ee8ebb12
Update MutateTask.cpp 2023-05-23 18:11:08 +03:00
Amos Bird
b82ff979d0
Fix invalid index analysis for date related keys 2023-05-23 23:10:34 +08:00
SmitaRKulkarni
55af60ea3f
Merge branch 'master' into 42192_Lower_parallel_parsing_threads_with_globs 2023-05-23 13:59:15 +02:00
Alexander Tokmakov
141a72d694
Merge pull request #49637 from ClickHouse/less_zookeeper_requests 
Provide better partitions hint for merge selecting task
 2023-05-23 12:40:39 +03:00
avogar
4f85d6a1bb Merge branch 'master' of github.com:ClickHouse/ClickHouse into random-structure 2023-05-22 19:43:24 +00:00
avogar
88e4c93abc Merge branch 'master' of github.com:ClickHouse/ClickHouse into urlCluster 2023-05-22 19:19:57 +00:00
kssenii
295fe3b228 Merge remote-tracking branch 'upstream/master' into abstract-async-prefetched-buffer 2023-05-22 20:01:38 +02:00
kssenii
c4d862a16f Make async reader work with any impl 2023-05-22 19:54:04 +02:00
Nikolay Degterinsky
d4b89cb643
Merge pull request #49356 from Ziy1-Tan/vcol 
Support for `_path` and `_file` virtual columns for table function `url`.
 2023-05-22 18:10:32 +02:00
Nikolay Degterinsky
7bed59e1d2
Merge pull request #50000 from evillique/add-schema-inference 
Add schema inference to more table engines
 2023-05-22 17:24:30 +02:00
Alexander Tokmakov
821b64b420 apply review suggestions 2023-05-22 15:18:29 +02:00
Kruglov Pavel
b5cad024e0
Merge branch 'master' into urlCluster 2023-05-22 14:59:34 +02:00
alesapin
cc3897a84a
Merge pull request #50033 from kssenii/disk-object-storage-minor-changes 
Get rid of indirect write buffer in object storages
 2023-05-22 14:08:17 +02:00
Alexander Tokmakov
487e510103 Merge branch 'master' into less_zookeeper_requests 2023-05-22 13:59:26 +02:00
Alexander Tokmakov
c89f92e1f6
Merge pull request #50052 from amosbird/fix_49913 
Fix reporting broken projection parts
 2023-05-22 14:20:06 +03:00
Antonio Andelic
93cd47fd5f
Merge pull request #50026 from ClickHouse/fix-deadlock-attach-thread 
Avoid deadlock when starting table in attach thread of `ReplicatedMergeTree`
 2023-05-22 09:22:33 +02:00
Amos Bird
0a3d986e42
Fix reporting projection broken part 2023-05-21 20:58:58 +08:00
Nikita Taranov
c93836b962 fix 2023-05-19 22:27:53 +00:00
kssenii
3e42ee7f2b Get rid of finalize callback in object storages 2023-05-19 17:29:37 +02:00
Antonio Andelic
4af8187464 Activate restarting thread in both cases 2023-05-19 15:06:02 +00:00
mateng915
5237dd0245
New system table zookeeper connection (#45245) 
* Feature: Support new system table to show which zookeeper node be connected

Description:
============
Currently we have no place to check which zk node be connected otherwise using
lsof command. It not convenient

Solution:
=========
Implemented a new system table, system.zookeeper_host when CK Server has zk
this table will show the zk node dir which connected by current CK server

Noted: This table can support multi-zookeeper cluster scenario.

* fixed review comments

* added test case

* update test cases

* remove unused code

* fixed review comments and removed unused code

* updated test cases for print host, port and is_expired

* modify the code comments

* fixed CI Failed

* fixed code style check failure

* updated test cases by added Tags

* update test reference

* update test cases

* added system.zookeeper_connection doc

* Update docs/en/operations/system-tables/zookeeper_connection.md

* Update docs/en/operations/system-tables/zookeeper_connection.md

* Update docs/en/operations/system-tables/zookeeper_connection.md

---------

Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2023-05-19 17:06:43 +03:00
Antonio Andelic
acf71c5b9a
Fix typo 2023-05-19 15:48:31 +02:00
Antonio Andelic
3107070e76 Avoid deadlock when starting table in attach thread 2023-05-19 12:48:19 +00:00
Nikolay Degterinsky
a09a8d60c4
Merge branch 'master' into postgresql-uuid 2023-05-19 13:44:17 +02:00
alesapin
e741450b88
Merge branch 'master' into fix_another_zero_copy_bug 2023-05-19 12:40:48 +02:00
alesapin
e5b001abda
Merge branch 'master' into fix_some_tests4 2023-05-19 12:34:03 +02:00
Michael Kolupaev
e84f0895e7 Support hardlinking parts transactionally 2023-05-18 21:05:56 -07:00
Nikolay Degterinsky
ef45956713 Fix style 2023-05-19 01:31:45 +00:00
Nikolay Degterinsky
b8be714830 Add schema inference to more table engines 2023-05-19 00:44:27 +00:00
Nikita Taranov
971cc092d4
Update src/Storages/MergeTree/MergeTreePrefetchedReadPool.cpp 2023-05-18 15:16:47 +02:00
SmitaRKulkarni
a91c793684
Merge branch 'master' into 42192_Lower_parallel_parsing_threads_with_globs 2023-05-18 09:24:25 +02:00
Kseniia Sumarokova
1c04085e8f
Update MergeTreeWriteAheadLog.h 2023-05-17 18:15:51 +02:00
Han Fei
ed1d036151
Merge pull request #49884 from azat/dist-fix-async-block-processing 
Fix processing pending batch for Distributed async INSERT after restart
 2023-05-17 15:19:42 +02:00
Nikolay Degterinsky
194ce2d881 Better 2023-05-17 13:13:57 +00:00
Nikita Taranov
0dd67bacf2
Merge branch 'master' into optimize_reading2 2023-05-17 15:06:41 +02:00
Alexander Tokmakov
36c31e1d79
Improve concurrent parts removal with zero copy replication (#49630) 
* improve concurrent parts removal

* fix

* fix
 2023-05-17 14:07:34 +03:00
Vitaly Baranov
6c8a923c9d
Merge branch 'master' into write-encrypted-to-backup 2023-05-17 12:37:05 +02:00
Vitaly Baranov
f4ac4c3f9d Corrections after review. 2023-05-17 03:23:16 +02:00
Alexander Tokmakov
242a3fc520 Merge branch 'master' into less_zookeeper_requests 2023-05-16 18:24:11 +02:00
Alexander Tokmakov
0da82945ac fix 2023-05-16 18:18:48 +02:00
Kruglov Pavel
4530f38fdf
Merge branch 'master' into urlCluster 2023-05-16 16:21:23 +02:00
alesapin
50a536bba8 Remove unused code 2023-05-16 15:26:24 +02:00
Alexander Tokmakov
b6716a8f0f Merge branch 'master' into fix_some_tests4 2023-05-16 14:46:27 +02:00
Vitaly Baranov
943707963f Add backup setting "decrypt_files_from_encrypted_disks" 2023-05-16 14:27:27 +02:00
Vitaly Baranov
5198997fd8 Remove ReadSettings from backup entries. 2023-05-16 14:27:27 +02:00
Vitaly Baranov
517e119e03 Move checksum calculation to IBackupEntry. 2023-05-16 14:27:27 +02:00
Vitaly Baranov
c92219f01b BACKUP now writes encrypted data for tables on encrypted disks. 2023-05-16 14:26:33 +02:00
Vitaly Baranov
cc50fcc60a Remove the 'temporary_file_' argument from BackupEntryFromImmutableFile's constructor. 2023-05-16 14:25:37 +02:00
alesapin
93bd09ddd6
Merge branch 'master' into fix_another_zero_copy_bug 2023-05-16 12:24:52 +02:00
alesapin
0b4ab70dd9
Merge pull request #49891 from hanfei1991/hanfei/chassert-1 
use chassert in MergeTreeDeduplicationLog to have better log info
 2023-05-16 11:50:11 +02:00
Sema Checherinda
03c51208d1
Merge pull request #44869 from CheSema/multi_part_upload 
rework WriteBufferFromS3, add tests, add abortion
 2023-05-16 10:52:01 +02:00
Nikolay Degterinsky
d9be88a36a Add UUID data type to PostgreSQL 2023-05-15 21:40:10 +00:00
Robert Schulze
59bc3e25be
Merge pull request #49824 from AVMusorin/allow-alias-column-kafka 
KafkaEngine: Allow usage of Alias column type
 2023-05-15 23:40:03 +02:00
Vitaly Baranov
801cacc13f
Merge pull request #49831 from vitlibar/fix-setting-null-in-profile-def 
Fix setting NULL in profile definition
 2023-05-15 22:24:49 +02:00
Alexander Tokmakov
65bc702b0b fix 2023-05-15 20:02:30 +02:00
Han Fei
4137a5e058 use chassert in MergeTreeDeduplicationLog to have better log info 2023-05-15 18:51:16 +02:00
avogar
334f062fa0 fix style 2023-05-15 16:39:26 +00:00
avogar
f9e0eb47d7 Merge branch 'master' of github.com:ClickHouse/ClickHouse into urlCluster 2023-05-15 16:35:24 +00:00
Alexander Tokmakov
05ae7b2c2d fix some tests 2023-05-15 18:28:12 +02:00
Azat Khuzhin
f2a023140e Fix processing pending batch for Distributed async INSERT after restart 
After abnormal server restart current_batch.txt (that contains list of
files to send to the remote shard), may not have all files, if it was
terminated between unlink .bin files and truncation of current_batch.txt

But it should be fixed in a more reliable way, though to backport the
patch I kept it simple.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-15 15:57:30 +02:00
AVMusorin
418a61a68c
Allow using Alias column type for KafkaEngine 
```
create table kafka
(
 a UInt32,
 a_str String Alias toString(a)
) engine = Kafka;

create table data
(
  a UInt32;
  a_str String
) engine = MergeTree
order by tuple();

create materialized view data_mv to data
(
  a UInt32,
  a_str String
) as
select a, a_str from kafka;
```
Alias type works as expected in comparison with MATERIALIZED/EPHEMERAL
or column with default expression.

Ref: https://github.com/ClickHouse/ClickHouse/pull/47138

Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com>
 2023-05-15 15:39:58 +02:00
Alexander Tokmakov
25912a2673
Merge pull request #49876 from JackyWoo/fix_typo 
fix typo
 2023-05-15 13:32:58 +03:00
Kruglov Pavel
558eda4146
Merge pull request #49412 from azat/block-use-dense-hash-map 
Switch Block::NameMap to google::dense_hash_map over HashMap
 2023-05-15 12:22:55 +02:00
JackyWoo
8d1bcb5c2f fix typo 2023-05-15 16:51:20 +08:00
Alexey Milovidov
b4b5b4dcc5 Amend 2023-05-14 02:52:23 +02:00
Alexey Milovidov
0ca36d4f89 Merge branch 'master' into clang-17 2023-05-14 01:57:40 +02:00
Smita Kulkarni
fd58eac75a Fixed max_threads datatype issue for builds 2023-05-13 10:22:37 +02:00
Alexey Milovidov
a2d1cc1333
Merge pull request #49844 from MikhailBurdukov/in_with_tuple 
TYPE_MISMATCH exception for in operator with  single column tuples.
 2023-05-13 04:21:12 +03:00
Alexey Milovidov
5a44dc26e7 Fixes for clang-17 2023-05-13 02:57:31 +02:00
Anton Popov
401030e2be
Merge pull request #49757 from CurtizJ/add-forgotten-lock 
Add forgotten lock (addition to #49117)
 2023-05-12 23:55:09 +02:00
MikhailBurdukov
8c7b63f1e7 Fix 2023-05-12 16:42:01 +00:00
Smita Kulkarni
792565d858 Updated to ULL 2023-05-12 17:23:37 +02:00
avogar
3d00f4cdf6 Remove unneded files 2023-05-12 14:03:04 +00:00
avogar
70a8fd2c50 Fix schema inference with named collection, refactor Cluster table functions 2023-05-12 13:58:45 +00:00
Robert Schulze
b9c185af44
Merge pull request #49678 from azat/build/llvm-16 
Switch to LLVM/clang 16 (16.0.3)
 2023-05-12 13:47:36 +02:00
Sema Checherinda
a1ee7d52e1
Merge pull request #49432 from CheSema/lost-blobs 
all s3-blobs removed when merge aborted, remove part from failed fetch without unlock keper
 2023-05-12 13:19:27 +02:00
Sema Checherinda
5706e05757 adding more comments 2023-05-12 13:00:56 +02:00
Nikita Taranov
536c5ac261
Merge branch 'master' into optimize_reading2 2023-05-12 12:59:32 +02:00
Alexander Gololobov
7c7565f094
Merge pull request #49737 from ClickHouse/fix_lwd_and_object 
Ignore LWD column in checkPartDynamicColumns
 2023-05-12 12:20:07 +02:00
Robert Schulze
922420420c
Merge pull request #49300 from ClickHouse/rs/functdocs 
Introduce more fields for in-source function documentation
 2023-05-12 11:36:04 +02:00
Vitaly Baranov
eb62030fa4 Fix assigning a setting to NULL in settings profile's definition. 2023-05-12 11:04:30 +02:00
Azat Khuzhin
2c40dd6a4c Switch Block::NameMap to google::dense_hash_map over HashMap 
Since HashMap creates 2^8 elements by default, while dense_hash_map
should be good here.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-12 05:52:57 +02:00
Alexey Milovidov
ea979b40a9
Merge pull request #49717 from ClickHouse/disable-mmap-for-server 
Disable mmap for StorageFile in clickhouse-server
 2023-05-11 23:53:01 +03:00
Anton Popov
d21dc829d9
Merge branch 'master' into add-forgotten-lock 2023-05-11 16:56:13 +02:00