Commit Graph

7101 Commits

Author SHA1 Message Date
Antonio Andelic
f5d3a8a31d Polishing 2022-03-11 13:38:19 +00:00
Antonio Andelic
28e9508c4e Improve shared pool and add settings 2022-03-11 08:33:34 +00:00
Antonio Andelic
cce318273b Use a shared IO thread pool 2022-03-11 08:33:34 +00:00
Antonio Andelic
df0f5e20d0 Refactor 2022-03-11 08:33:34 +00:00
Antonio Andelic
e051587fc2 Format code 2022-03-11 08:33:34 +00:00
Antonio Andelic
29c32ed831 Refactor code 2022-03-11 08:33:34 +00:00
Antonio Andelic
0a1a3a230e Add support for parallel http range requests 2022-03-11 08:33:34 +00:00
Antonio Andelic
58557b9bec Test check for ranges support 2022-03-11 08:33:34 +00:00
Kseniia Sumarokova
818459b9f0
Merge pull request #33717 from kssenii/local-cache-for-remote-fs
Local cache for remote filesystem
2022-03-11 07:23:10 +01:00
alesapin
c0d8ccc91b
Merge pull request #35178 from Varinara/master
Added disk_name to system.part_log
2022-03-10 22:22:37 +01:00
Maksim Kita
493169910b
Merge pull request #35174 from zhangyifan27/fix_typo
fix typos
2022-03-10 17:10:44 +01:00
Kseniia Sumarokova
3fc399b6e9
Merge pull request #35158 from kssenii/fix-materialized-postgresql
Fix materialised postrgesql adding new table after manually removing it
2022-03-10 17:02:32 +01:00
Kseniia Sumarokova
e30b0c5d57
Merge pull request #35162 from kssenii/fix-materialized-postgresql-table-override
Fix materialised postgres `table overrides` for partition by, etc
2022-03-10 17:01:24 +01:00
Varinara
f5523f7ff0 added disk_name to system.part_log 2022-03-10 18:44:19 +03:00
mergify[bot]
df01290e73
Merge branch 'master' into fix_typo 2022-03-10 13:35:04 +00:00
kssenii
1dc3f36a11 Better 2022-03-10 12:19:20 +01:00
kssenii
787a0805a5 Merge master 2022-03-10 11:42:19 +01:00
Kseniia Sumarokova
e6ee891c9c
Merge pull request #34957 from bigo-sg/hive_random_access_file_cache
Optimization for first time to read a random access readbuffer in hive
2022-03-10 11:36:22 +01:00
zhangyifan27
e6fa9f699a fix typo 2022-03-10 18:29:42 +08:00
kssenii
3cd1da1e11 Fix 2022-03-10 11:11:59 +01:00
kssenii
af9d8d278e Fix 2022-03-09 19:25:43 +01:00
Vladimir C
ce266b5a3e
Merge pull request #35146 from amosbird/fixpartitionprunerin 2022-03-09 13:23:45 +01:00
Nikolai Kochetov
6bfee7aca2
Merge pull request #35132 from azat/parallel_distributed_insert_select-view
Support view() for parallel_distributed_insert_select
2022-03-09 09:10:34 +01:00
Amos Bird
a19224bc9b
Fix partition pruner: non-monotonic function IN 2022-03-09 15:48:42 +08:00
Azat Khuzhin
4843e210c3 Support view() for parallel_distributed_insert_select
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-08 22:05:57 +03:00
Azat Khuzhin
ced34dea84 Take flush_time into account for scheduling background flush of the Buffer
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-08 21:58:10 +03:00
kssenii
5260822964 Merge master 2022-03-08 18:21:28 +01:00
kssenii
e231c3a3e0 Fix split build 2022-03-08 18:05:55 +01:00
Azat Khuzhin
c4b6342853
Improvements for parallel_distributed_insert_select (and related) (#34728)
* Add a warning if parallel_distributed_insert_select was ignored

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Respect max_distributed_depth for parallel_distributed_insert_select

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Print warning for non applied parallel_distributed_insert_select only for initial query

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Remove Cluster::getHashOfAddresses()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Forbid parallel_distributed_insert_select for remote()/cluster() with different addresses

Before it uses empty cluster name (getClusterName()) which is not
correct, compare all addresses instead.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Fix max_distributed_depth check

max_distributed_depth=1 must mean not more then one distributed query,
not two, since max_distributed_depth=0 means no limit, and
distribute_depth is 0 for the first query.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Fix INSERT INTO remote()/cluster() with parallel_distributed_insert_select

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Add a test for parallel_distributed_insert_select with cluster()/remote()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Return <remote> instead of empty cluster name in Distributed engine

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Make user with sharding_key and w/o in remote()/cluster() identical

Before with sharding_key the user was "default", while w/o it it was
empty.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-08 15:24:39 +01:00
Antonio Andelic
bc5d7aea57
Merge pull request #34876 from azat/long-INSERT-fix
Fix possible "Part directory doesn't exist" during INSERT
2022-03-08 12:44:53 +01:00
Kseniia Sumarokova
1eb2bae792
Merge pull request #34954 from bigo-sg/hive_read_columns_pruning
read columns pruning for hive
2022-03-08 10:17:24 +01:00
lgbo-ustc
256e92ffee Merge remote-tracking branch 'ck/master' into hive_random_access_file_cache 2022-03-08 14:14:40 +08:00
Azat Khuzhin
caffc144b5 Fix possible "Part directory doesn't exist" during INSERT
In #33291 final part commit had been defered, and now it can take
significantly more time, that may lead to "Part directory doesn't exist"
error during INSERT:

    2022.02.21 18:18:06.979881 [ 11329 ] {insert} <Debug> executeQuery: (from 127.1:24572, user: default) INSERT INTO db.table (...) VALUES
    2022.02.21 20:58:03.933593 [ 11329 ] {insert} <Trace> db.table: Renaming temporary part tmp_insert_20220214_18044_18044_0 to 20220214_270654_270654_0.
    2022.02.21 21:16:50.961917 [ 11329 ] {insert} <Trace> db.table: Renaming temporary part tmp_insert_20220214_18197_18197_0 to 20220214_270689_270689_0.
    ...
    2022.02.22 21:16:57.632221 [ 64878 ] {} <Warning> db.table: Removing temporary directory /clickhouse/data/db/table/tmp_insert_20220214_18232_18232_0/
    ...
    2022.02.23 12:23:56.277480 [ 11329 ] {insert} <Trace> db.table: Renaming temporary part tmp_insert_20220214_18232_18232_0 to 20220214_273459_273459_0.
    2022.02.23 12:23:56.299218 [ 11329 ] {insert} <Error> executeQuery: Code: 107. DB::Exception: Part directory /clickhouse/data/db/table/tmp_insert_20220214_18232_18232_0/ doesn't exist. Most likely it is a logical error. (FILE_DOESNT_EXIST) (version 22.2.1.1) (from 127.1:24572) (in query: INSERT INTO db.table (...) VALUES), Stack trace (when copying this message, always include the lines below):

Follow-up for: #28760
Refs: #33291

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-08 07:44:11 +03:00
lgbo-ustc
a8cfc2458a update codes 2022-03-08 11:55:15 +08:00
Maksim Kita
2f9361008b
Merge pull request #35089 from 1lann/1lann/fix-update_lag-typo
Fix typo of update_lag
2022-03-07 23:12:35 +01:00
Kseniia Sumarokova
5511f2f6e6
Merge pull request #34940 from bigo-sg/hive_client_connection_pool
Use connection pool in HiveMetastoreClient
2022-03-07 17:14:56 +01:00
Kseniia Sumarokova
28b9ec01c0
Merge pull request #34945 from bigo-sg/hive_bug_fixed
unexpected result when use `in` in hive query
2022-03-07 17:13:11 +01:00
alesapin
d90bee1df9
Merge pull request #35080 from azat/mutate-exceptions
Do not hide exceptions during mutations
2022-03-07 12:22:49 +01:00
Vladimir C
678f05ca1e
Merge pull request #34912 from kssenii/fix-filelog-metadata-path 2022-03-07 11:45:29 +01:00
1lann
5423c5a45c Fix typo of update_lag
In external dictionary providers, the allowed keys for configuration seemed to have a typo
of "update_lag" as "update_tag", preventing the use of "update_lag". This change fixes that.
2022-03-07 18:31:20 +08:00
lgbo-ustc
8ae5296ee8 fixed compile errors 2022-03-07 17:26:48 +08:00
lgbo-ustc
eab925554d fixed code styles 2022-03-07 14:01:03 +08:00
lgbo-ustc
0c83b96d8c fixed code style 2022-03-07 13:13:33 +08:00
lgbo-ustc
cfeedd2cb5 fixed code style 2022-03-07 12:28:31 +08:00
lgbo-ustc
4507cc58aa update codes 2022-03-07 12:05:07 +08:00
lgbo-ustc
c37eedd887 update codes 2022-03-07 10:30:54 +08:00
lgbo-ustc
75a50a30c4 update codes 2022-03-07 09:43:53 +08:00
lgbo-ustc
d907b70cc4 update codes: get actual read block 2022-03-07 09:26:05 +08:00
lgbo-ustc
f4d8fb46c5 update codes 2022-03-07 09:26:05 +08:00
lgbo-ustc
62c1bd5ae9 hive read columns pruning 2022-03-07 09:26:05 +08:00