mergify[bot]
9b9959aae0
Merge branch 'master' into read_in_order_max_rows_to_read
2022-03-21 17:33:37 +00:00
Vitaly Baranov
72a2f31984
Merge pull request #33985 from vitlibar/improve-backup-and-restore
...
Improve backup and restore (experimental)
2022-03-21 17:59:51 +01:00
tavplubix
9226ea7b70
Merge pull request #35467 from ClickHouse/resubmit_21474
...
Resubmit #21474
2022-03-21 19:15:28 +03:00
Nikolai Kochetov
5e239762c6
Merge pull request #35334 from amosbird/fixpartitionpruneroverflow
...
Fix incorrect partition pruning when constant in predicate has no supertype of related key columns
2022-03-21 17:05:19 +01:00
Alexander Tokmakov
dba82a7732
fix show create for information_schema
2022-03-21 17:01:39 +01:00
Antonio Andelic
04e6e6ab5e
Add ParallelReadBuffer for S3
2022-03-21 14:52:26 +00:00
kssenii
afd0c64a1a
Better
2022-03-21 15:04:16 +01:00
mergify[bot]
e11ef05c2b
Merge branch 'master' into issue_33147
2022-03-21 13:40:17 +00:00
lgbo-ustc
967d5a8055
Merge remote-tracking branch 'ck/master' into hive_column_pruning_bug
2022-03-21 19:52:06 +08:00
kssenii
eabbce69a7
Add system drop remote filesystem cache query, allow to write cache on insert
2022-03-21 12:32:44 +01:00
Alexander Tokmakov
7cba97aeab
Merge branch 'master' into resubmit_21474
2022-03-21 12:09:00 +01:00
Vitaly Baranov
7a46e7a0d1
Merge branch 'master' into improve-backup-and-restore
2022-03-21 11:49:26 +01:00
Vitaly Baranov
51fc556fcd
Implement BACKUP and RESTORE for MatViews.
2022-03-21 11:45:21 +01:00
mergify[bot]
e5a5ab2a40
Merge branch 'master' into distributedmultiplejoin
2022-03-21 10:00:51 +00:00
Antonio Andelic
b8c43ff2f2
Merge branch 'master' into parallel-downloading-url-engine
2022-03-21 09:11:01 +00:00
kssenii
59b7394caf
WriteSettings
2022-03-21 09:52:48 +01:00
kssenii
1d80262a2a
Add write settings, file cache settings
2022-03-21 09:50:50 +01:00
taiyang-li
49b6f3dfc5
merge master and fix conflict
2022-03-21 15:05:43 +08:00
taiyang-li
bf05b94940
fix build
2022-03-21 15:03:28 +08:00
taiyang-li
d2bea0a72c
Merge branch 'master' into improve_access_type
2022-03-21 14:25:19 +08:00
Alexey Milovidov
0ce4696c49
Revert "[WIP] New row policies"
2022-03-21 08:41:33 +03:00
taiyang-li
7d50bd1eb3
add access type hive
2022-03-21 11:19:45 +08:00
Amos Bird
243de091bb
Validate some thoughts over making sets
2022-03-21 10:58:44 +08:00
lgbo-ustc
f7aa40af5b
update codes
2022-03-21 09:25:20 +08:00
Alexander Tokmakov
9e05b12d2c
Merge branch 'master' into mvcc_prototype
2022-03-20 22:42:26 +01:00
Vitaly Baranov
ce25afb2e9
Storages and databases are hollow by default now.
2022-03-20 20:02:15 +01:00
Vitaly Baranov
c884cf901f
Use BackupEntryFromAppendOnlyFile for Log engine,
...
and use a BackupEntriesBatch for Memort engine to improve performance.
A lot of minor corrections.
2022-03-20 20:02:15 +01:00
Vitaly Baranov
e72a343994
Implement BACKUP & RESTORE for the Memory table engine.
2022-03-20 20:02:15 +01:00
Vitaly Baranov
7f89b98308
Rework BackupSettings and RestoreSettings a little, pass StorageRestoreSettings to storages.
2022-03-20 20:02:15 +01:00
Vitaly Baranov
258a472001
Shorter names: rename IRestoreFromBackupTask -> IRestoreTask.
2022-03-20 20:02:15 +01:00
Vitaly Baranov
5a883c6b5a
Fixed checking access rights while producing backup, add IStorage::hasHollowBackup().
2022-03-20 20:01:54 +01:00
Vitaly Baranov
7a63feb3f7
Make restore tasks explicit.
2022-03-20 20:01:31 +01:00
Vitaly Baranov
9fb50422ce
Merge pull request #35345 from vitlibar/new-row-policies
...
[WIP] New row policies
2022-03-20 20:00:25 +01:00
Alexey Milovidov
9b05801e0f
Merge pull request #35284 from CurtizJ/fix-mutations-sparse-columns
...
Fix mutations in tables with enabled sparse columns
2022-03-20 21:55:59 +03:00
mergify[bot]
7ac606fa65
Merge branch 'master' into parallel-multipart-upload-for-s3storage
2022-03-20 18:25:28 +00:00
alesapin
1bd5736e34
Fix build one more time
2022-03-19 20:00:08 +01:00
alesapin
9e24677a30
Fix build
2022-03-19 18:09:01 +01:00
alesapin
97e84e6dc2
fix build
2022-03-19 17:47:35 +01:00
alesapin
f2c5e2d3a0
Don't spam logs in zero copy replication
2022-03-19 17:31:33 +01:00
Kseniia Sumarokova
a918e68d9a
Merge pull request #35355 from kssenii/add-hosts-filter
...
Add more checks with remoteHostsFilter
2022-03-19 09:41:23 +01:00
Alexander Tokmakov
418d52044a
try fix merge assignment
2022-03-18 20:31:44 +01:00
Alexander Tokmakov
c28008a2df
more comments, minor fixes
2022-03-18 12:01:26 +01:00
Amos Bird
4d85c2fc8b
Use castColumnAccurateOrNull
2022-03-18 17:47:04 +08:00
Amos Bird
10996b1434
Fix mixed constant type during partition pruning
2022-03-18 17:47:03 +08:00
lgbo-ustc
e78cfe3b26
update codes
2022-03-18 15:07:52 +08:00
Alexander Tokmakov
07d952b728
use snapshots for semistructured data, durability fixes
2022-03-17 18:26:18 +01:00
Antonio Andelic
b789a41662
Fix ASAN problems
2022-03-17 15:18:20 +00:00
Anton Popov
11ab24acb1
Merge pull request #35356 from CurtizJ/fix-clang-tidy
...
Fix clang-tidy
2022-03-17 16:17:23 +01:00
Vitaly Baranov
470582e262
Change type of RowPolicyKind: bool -> enum.
2022-03-17 15:28:05 +01:00
Alexander Tokmakov
d04dc03fa4
Merge branch 'master' into mvcc_prototype
2022-03-17 15:24:32 +01:00
Nikolai Kochetov
ee9c2ec735
Merge pull request #34780 from azat/mt-delayed-part-flush
...
Do not delay final part writing by default (fixes possible Memory limit exceeded during INSERT)
2022-03-17 12:30:51 +01:00
Anton Popov
fe2dc85788
fix clang-tidy
2022-03-17 11:16:32 +00:00
kssenii
a2cd165d38
Add remote host filter
2022-03-17 11:48:42 +01:00
alesapin
92ab33f560
Merge pull request #35348 from ClickHouse/fix_yet_another_zero_copy_bug
...
Fix race in zero copy replication
2022-03-17 11:34:38 +01:00
alesapin
bb251938dc
Merge pull request #35344 from ClickHouse/changelog-22.3
...
Changelog 22.3
2022-03-17 11:25:36 +01:00
Antonio Andelic
103a3fa140
Merge branch 'master' into parallel-downloading-url-engine
2022-03-17 10:11:58 +00:00
Antonio Andelic
44de5a581c
Fix split build
2022-03-17 10:09:16 +00:00
Antonio Andelic
cc623bbf5e
Fix redirect_url tests
2022-03-17 08:03:22 +00:00
Alexey Milovidov
68ef49ea51
Fix something stupid
2022-03-17 05:57:13 +01:00
alesapin
03ff6d52f6
Fix race in zero copy replication
2022-03-16 21:51:33 +01:00
Anton Popov
de2cc23e15
fix race
2022-03-16 20:16:59 +00:00
Alexander Tokmakov
4b3e13a4fe
Merge branch 'master' into mvcc_prototype
2022-03-16 21:06:19 +01:00
Alexander Tokmakov
c2ac8d4a5c
review fixes
2022-03-16 21:05:34 +01:00
Sergei Trifonov
bbe657ba24
allow parallelization of multipart S3 storage upload fixes #34350
2022-03-16 19:41:41 +01:00
Antonio Andelic
0979155f2f
Address PR comments
2022-03-16 15:45:17 +00:00
Anton Popov
0ba78c3c3a
Merge remote-tracking branch 'upstream/master' into HEAD
2022-03-16 15:28:09 +00:00
Antonio Andelic
84f86bf3da
Limit number of bytes read per worker
2022-03-16 12:34:14 +00:00
Antonio Andelic
2291b7c1e1
Merge branch 'master' into parallel-downloading-url-engine
2022-03-16 09:59:52 +00:00
Nikolai Kochetov
a380aa6b8a
Merge pull request #35294 from ClickHouse/reload-remote_url_allow_hosts
...
Reload remote_url_allow_hosts after config update.
2022-03-15 22:07:16 +01:00
Alexander Tokmakov
1f571b7734
Merge branch 'master' into mvcc_prototype
2022-03-15 14:45:06 +01:00
Nikolai Kochetov
97aa6c82ce
Reload remote_url_allow_hosts after config update.
2022-03-15 13:00:31 +00:00
Anton Popov
b5a04ff938
Merge pull request #35207 from bigo-sg/add_distributed_depth_in_querylog
...
Add column distributed_depth for system.query_log and system.processes
2022-03-15 13:38:21 +01:00
Kseniia Sumarokova
427ae808ba
Merge pull request #35191 from DevTeamBK/RemoteHostFilter_improvement
...
Added RemoteHostFilter check for MYSQL and postgresSQL
2022-03-15 09:58:08 +01:00
Antonio Andelic
067b79b00b
Merge branch 'master' into parallel-downloading-url-engine
2022-03-15 07:55:41 +00:00
lgbo-ustc
abfaa82bca
fixed hive query bugs
2022-03-15 12:01:34 +08:00
Anton Popov
ccbddd53a3
fix mutations in tables with enabled sparse columns
2022-03-15 01:48:21 +00:00
alesapin
fbb1ebd9b8
Merge pull request #35274 from CurtizJ/fix-check-table-sparse-columns
...
Fix check table in case when there exist sparse columns
2022-03-14 21:56:04 +01:00
Alexander Tokmakov
9702b5177d
Merge branch 'master' into mvcc_prototype
2022-03-14 21:45:38 +01:00
Alexander Tokmakov
278d779a01
log cleanup, more comments
2022-03-14 21:43:34 +01:00
Maksim Kita
2fdcf53a76
Fix clang-tidy warnings in Server, Storages folders
2022-03-14 18:17:35 +00:00
Anton Popov
063917786e
minor fixes
2022-03-14 17:29:18 +00:00
Anton Popov
36ec379aeb
Merge remote-tracking branch 'upstream/master' into HEAD
2022-03-14 16:28:35 +00:00
Anton Popov
428bbd6377
fix check table in case when there exist sparse columns
2022-03-14 15:22:23 +00:00
Heena Bansal
c774458f96
Update src/Storages/StoragePostgreSQL.cpp
...
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
2022-03-14 10:38:12 -04:00
Heena Bansal
01c8b2f71e
Update src/Storages/StorageMySQL.cpp
...
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
2022-03-14 10:37:59 -04:00
Antonio Andelic
d3353f3f0c
Merge branch 'master' into parallel-downloading-url-engine
2022-03-14 12:47:23 +00:00
Antonio Andelic
556fe2bcc5
Don't send RANGE with HEAD call
2022-03-14 12:46:22 +00:00
Antonio Andelic
ed3d71d83f
Merge pull request #35072 from azat/buffer-memory-tracker-leak
...
Avoid MEMORY_LIMIT_EXCEEDED during INSERT into Buffer with AggregateFunction
2022-03-14 12:31:04 +01:00
Maksim Kita
ad6b3693e1
Merge pull request #35123 from zhanghuajieHIT/fix_build_fail_with_gcc
...
fix build fail with gcc
2022-03-14 10:36:15 +01:00
Antonio Andelic
9dda2863d3
Split download threads when multiple URLs are used
2022-03-14 09:27:09 +00:00
HeenaBansal2009
09be30ac26
Added suggestion from review comments
2022-03-11 07:33:23 -08:00
zhanghuajie
53a8987b3b
fix build fail with gcc --fix warnings without disabling some parameters
2022-03-11 21:59:19 +08:00
Antonio Andelic
f5d3a8a31d
Polishing
2022-03-11 13:38:19 +00:00
taiyang-li
1f9e050152
add column distributed_depth for system.query_log and system.processes
2022-03-11 17:57:34 +08:00
Antonio Andelic
28e9508c4e
Improve shared pool and add settings
2022-03-11 08:33:34 +00:00
Antonio Andelic
cce318273b
Use a shared IO thread pool
2022-03-11 08:33:34 +00:00
Antonio Andelic
df0f5e20d0
Refactor
2022-03-11 08:33:34 +00:00
Antonio Andelic
e051587fc2
Format code
2022-03-11 08:33:34 +00:00
Antonio Andelic
29c32ed831
Refactor code
2022-03-11 08:33:34 +00:00
Antonio Andelic
0a1a3a230e
Add support for parallel http range requests
2022-03-11 08:33:34 +00:00
Antonio Andelic
58557b9bec
Test check for ranges support
2022-03-11 08:33:34 +00:00
Kseniia Sumarokova
818459b9f0
Merge pull request #33717 from kssenii/local-cache-for-remote-fs
...
Local cache for remote filesystem
2022-03-11 07:23:10 +01:00
Alexander Tokmakov
7f47f20aba
add kill transaction query
2022-03-10 22:29:58 +01:00
Anton Popov
37efe2ddb5
Apply suggestions from code review
...
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
2022-03-10 22:24:19 +01:00
alesapin
c0d8ccc91b
Merge pull request #35178 from Varinara/master
...
Added disk_name to system.part_log
2022-03-10 22:22:37 +01:00
HeenaBansal2009
d981463b05
Added RemoteHostFilter check for MYSQL and postgresSQL
2022-03-10 08:58:48 -08:00
Maksim Kita
493169910b
Merge pull request #35174 from zhangyifan27/fix_typo
...
fix typos
2022-03-10 17:10:44 +01:00
Kseniia Sumarokova
3fc399b6e9
Merge pull request #35158 from kssenii/fix-materialized-postgresql
...
Fix materialised postrgesql adding new table after manually removing it
2022-03-10 17:02:32 +01:00
Kseniia Sumarokova
e30b0c5d57
Merge pull request #35162 from kssenii/fix-materialized-postgresql-table-override
...
Fix materialised postgres `table overrides` for partition by, etc
2022-03-10 17:01:24 +01:00
Varinara
f5523f7ff0
added disk_name to system.part_log
2022-03-10 18:44:19 +03:00
Rajkumar
3d3b6d1956
clang-tidy report issues with Medium priority
2022-03-10 07:23:49 -08:00
mergify[bot]
df01290e73
Merge branch 'master' into fix_typo
2022-03-10 13:35:04 +00:00
Alexander Tokmakov
061fa6a6f2
Merge branch 'master' into mvcc_prototype
2022-03-10 13:13:04 +01:00
kssenii
1dc3f36a11
Better
2022-03-10 12:19:20 +01:00
kssenii
787a0805a5
Merge master
2022-03-10 11:42:19 +01:00
Kseniia Sumarokova
e6ee891c9c
Merge pull request #34957 from bigo-sg/hive_random_access_file_cache
...
Optimization for first time to read a random access readbuffer in hive
2022-03-10 11:36:22 +01:00
zhangyifan27
e6fa9f699a
fix typo
2022-03-10 18:29:42 +08:00
kssenii
3cd1da1e11
Fix
2022-03-10 11:11:59 +01:00
Alexander Tokmakov
0906b59fba
fixes
2022-03-09 21:38:18 +01:00
kssenii
af9d8d278e
Fix
2022-03-09 19:25:43 +01:00
Vladimir C
ce266b5a3e
Merge pull request #35146 from amosbird/fixpartitionprunerin
2022-03-09 13:23:45 +01:00
Nikolai Kochetov
6bfee7aca2
Merge pull request #35132 from azat/parallel_distributed_insert_select-view
...
Support view() for parallel_distributed_insert_select
2022-03-09 09:10:34 +01:00
Azat Khuzhin
e2960e1a52
Avoid MEMORY_LIMIT_EXCEEDED during INSERT into Buffer with AggregateFunction
...
In case of Buffer table has columns of AggregateFunction type,
aggregate states for such columns will be allocated from the query
context but those states can be destroyed from the server context (in
case of background flush), and thus memory will be leaked from the query
since aggregate states can be shared, and eventually this will lead to
MEMORY_LIMIT_EXCEEDED error.
To avoid this, prohibit sharing the aggregate states.
But note, that this problem only about memory accounting, not memory
usage itself.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-09 10:57:49 +03:00
Amos Bird
a19224bc9b
Fix partition pruner: non-monotonic function IN
2022-03-09 15:48:42 +08:00
Azat Khuzhin
3a5a39a9df
Do not delay final part writing by default
...
For async s3 writes final part flushing was defered until all the INSERT
block was processed, however in case of too many partitions/columns you
may exceed max_memory_usage limit (since each stream has overhead).
Introduce max_insert_delayed_streams_for_parallel_writes (with default
to 1000 for S3, 0 otherwise), to avoid this.
This should "Memory limit exceeded" errors in performance tests.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-08 22:17:36 +03:00
Alexander Tokmakov
d2f838bd91
fix another race condition
2022-03-08 20:11:47 +01:00
Azat Khuzhin
4843e210c3
Support view() for parallel_distributed_insert_select
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-08 22:05:57 +03:00
Azat Khuzhin
ced34dea84
Take flush_time into account for scheduling background flush of the Buffer
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-08 21:58:10 +03:00
kssenii
5260822964
Merge master
2022-03-08 18:21:28 +01:00
kssenii
e231c3a3e0
Fix split build
2022-03-08 18:05:55 +01:00
Azat Khuzhin
c4b6342853
Improvements for parallel_distributed_insert_select (and related) ( #34728 )
...
* Add a warning if parallel_distributed_insert_select was ignored
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
* Respect max_distributed_depth for parallel_distributed_insert_select
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
* Print warning for non applied parallel_distributed_insert_select only for initial query
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
* Remove Cluster::getHashOfAddresses()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
* Forbid parallel_distributed_insert_select for remote()/cluster() with different addresses
Before it uses empty cluster name (getClusterName()) which is not
correct, compare all addresses instead.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
* Fix max_distributed_depth check
max_distributed_depth=1 must mean not more then one distributed query,
not two, since max_distributed_depth=0 means no limit, and
distribute_depth is 0 for the first query.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
* Fix INSERT INTO remote()/cluster() with parallel_distributed_insert_select
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
* Add a test for parallel_distributed_insert_select with cluster()/remote()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
* Return <remote> instead of empty cluster name in Distributed engine
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
* Make user with sharding_key and w/o in remote()/cluster() identical
Before with sharding_key the user was "default", while w/o it it was
empty.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-08 15:24:39 +01:00
Antonio Andelic
bc5d7aea57
Merge pull request #34876 from azat/long-INSERT-fix
...
Fix possible "Part directory doesn't exist" during INSERT
2022-03-08 12:44:53 +01:00
Kseniia Sumarokova
1eb2bae792
Merge pull request #34954 from bigo-sg/hive_read_columns_pruning
...
read columns pruning for hive
2022-03-08 10:17:24 +01:00
lgbo-ustc
256e92ffee
Merge remote-tracking branch 'ck/master' into hive_random_access_file_cache
2022-03-08 14:14:40 +08:00
Azat Khuzhin
caffc144b5
Fix possible "Part directory doesn't exist" during INSERT
...
In #33291 final part commit had been defered, and now it can take
significantly more time, that may lead to "Part directory doesn't exist"
error during INSERT:
2022.02.21 18:18:06.979881 [ 11329 ] {insert} <Debug> executeQuery: (from 127.1:24572, user: default) INSERT INTO db.table (...) VALUES
2022.02.21 20:58:03.933593 [ 11329 ] {insert} <Trace> db.table: Renaming temporary part tmp_insert_20220214_18044_18044_0 to 20220214_270654_270654_0.
2022.02.21 21:16:50.961917 [ 11329 ] {insert} <Trace> db.table: Renaming temporary part tmp_insert_20220214_18197_18197_0 to 20220214_270689_270689_0.
...
2022.02.22 21:16:57.632221 [ 64878 ] {} <Warning> db.table: Removing temporary directory /clickhouse/data/db/table/tmp_insert_20220214_18232_18232_0/
...
2022.02.23 12:23:56.277480 [ 11329 ] {insert} <Trace> db.table: Renaming temporary part tmp_insert_20220214_18232_18232_0 to 20220214_273459_273459_0.
2022.02.23 12:23:56.299218 [ 11329 ] {insert} <Error> executeQuery: Code: 107. DB::Exception: Part directory /clickhouse/data/db/table/tmp_insert_20220214_18232_18232_0/ doesn't exist. Most likely it is a logical error. (FILE_DOESNT_EXIST) (version 22.2.1.1) (from 127.1:24572) (in query: INSERT INTO db.table (...) VALUES), Stack trace (when copying this message, always include the lines below):
Follow-up for: #28760
Refs: #33291
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-08 07:44:11 +03:00
lgbo-ustc
a8cfc2458a
update codes
2022-03-08 11:55:15 +08:00
taiyang-li
b4174b0bef
merge master and fix conflicts
2022-03-08 11:39:25 +08:00
Maksim Kita
2f9361008b
Merge pull request #35089 from 1lann/1lann/fix-update_lag-typo
...
Fix typo of update_lag
2022-03-07 23:12:35 +01:00
Alexander Tokmakov
8acfb8d27f
Merge branch 'master' into mvcc_prototype
2022-03-07 17:40:15 +01:00
Alexander Tokmakov
ea2f65fef6
fix tests with DiskS3
2022-03-07 17:35:47 +01:00
Kseniia Sumarokova
5511f2f6e6
Merge pull request #34940 from bigo-sg/hive_client_connection_pool
...
Use connection pool in HiveMetastoreClient
2022-03-07 17:14:56 +01:00
Kseniia Sumarokova
28b9ec01c0
Merge pull request #34945 from bigo-sg/hive_bug_fixed
...
unexpected result when use `in` in hive query
2022-03-07 17:13:11 +01:00
Anton Popov
0bc57da238
Merge remote-tracking branch 'upstream/master' into HEAD
2022-03-07 14:46:08 +00:00
alesapin
d90bee1df9
Merge pull request #35080 from azat/mutate-exceptions
...
Do not hide exceptions during mutations
2022-03-07 12:22:49 +01:00
Vladimir C
678f05ca1e
Merge pull request #34912 from kssenii/fix-filelog-metadata-path
2022-03-07 11:45:29 +01:00
1lann
5423c5a45c
Fix typo of update_lag
...
In external dictionary providers, the allowed keys for configuration seemed to have a typo
of "update_lag" as "update_tag", preventing the use of "update_lag". This change fixes that.
2022-03-07 18:31:20 +08:00
lgbo-ustc
8ae5296ee8
fixed compile errors
2022-03-07 17:26:48 +08:00
lgbo-ustc
eab925554d
fixed code styles
2022-03-07 14:01:03 +08:00
lgbo-ustc
0c83b96d8c
fixed code style
2022-03-07 13:13:33 +08:00
lgbo-ustc
cfeedd2cb5
fixed code style
2022-03-07 12:28:31 +08:00
lgbo-ustc
4507cc58aa
update codes
2022-03-07 12:05:07 +08:00
lgbo-ustc
c37eedd887
update codes
2022-03-07 10:30:54 +08:00
lgbo-ustc
75a50a30c4
update codes
2022-03-07 09:43:53 +08:00
lgbo-ustc
d907b70cc4
update codes: get actual read block
2022-03-07 09:26:05 +08:00
lgbo-ustc
f4d8fb46c5
update codes
2022-03-07 09:26:05 +08:00
lgbo-ustc
62c1bd5ae9
hive read columns pruning
2022-03-07 09:26:05 +08:00
Azat Khuzhin
bc224dee36
Do not hide exceptions during mutations
...
system.mutations includes only the message, but not stacktrace, and it
is not always obvious to understand the culprit w/o stacktrace.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-06 13:39:49 +03:00
Kseniia Sumarokova
3ec6cd3128
Update StorageFileLog.cpp
2022-03-06 11:03:22 +01:00
alexey-milovidov
f9b7df6ba1
Merge pull request #35050 from CurtizJ/fix-async-inserts-system-table
...
Fix reading from `system.asynchronous_inserts` table
2022-03-06 02:25:53 +03:00
Maksim Kita
7ae1f0fa3b
Merge pull request #34911 from larspars/master
...
Allow LowCardinality strings for ngrambf_v1/tokenbf_v1 indexes. Fixes #21865
2022-03-04 19:17:48 +01:00
Azat Khuzhin
2ef9d32448
Revert "Remove VERSION_DATE from system.build_options"
...
As requested by @kitaisreal
This reverts commit 4a404532fb
2022-03-04 15:32:54 +03:00
Azat Khuzhin
4a404532fb
Remove VERSION_DATE from system.build_options
...
It was set only bu utils/release/release_lib.sh, and seems that this
script is not used anymore, at least that part of it.
Also note, that GIT_DATE is the same, and it is date time, not only
date.
Plus VERSION_DATE is not installed for releases anyway.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-04 15:31:37 +03:00
Azat Khuzhin
b0f964a14f
Remove LIBRARY_ARCHITECTURE from system.build_options
...
CMAKE_LIBRARY_ARCHITECTURE and it is useless, since it is reported only
if the compiler reports subdir arch triplet [1]
[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1531678
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-04 15:31:32 +03:00
Azat Khuzhin
494fe91f86
Fix LINK_FLAGS in system.build_options
...
Fixes: 79f6f5a202
2022-03-04 15:31:32 +03:00
Azat Khuzhin
c426eef07d
Fix generating USE_* for system.build_options
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-04 15:31:32 +03:00
Anton Popov
c836a57000
fix reading from system.asynchronous_inserts table
2022-03-04 11:46:15 +00:00
Anton Popov
df3b07fe7c
Merge remote-tracking branch 'upstream/master' into HEAD
2022-03-03 22:25:28 +00:00
Anton Popov
aea7bfb59a
Merge pull request #34992 from azat/fix-asynchronous_inserts-race
...
Fix race between INSERT async_insert=1 and system.asynchronous_inserts
2022-03-03 20:55:19 +03:00
kssenii
d19f199e93
Revert
2022-03-03 15:25:27 +01:00
Kseniia Sumarokova
b11b34dc8c
Merge pull request #34849 from kssenii/fix-too-many-columns
...
Fix reading too many columns for s3 and url storages
2022-03-03 13:57:22 +01:00
Kseniia Sumarokova
ad09554c4c
Merge pull request #34996 from kssenii/fix-filelog-assertion
...
Fix possible segfault in filelog storage
2022-03-03 13:52:54 +01:00
Frank Chen
b4829465d9
Improve the opentelemetry span logs for INSERT on distributed table ( #34480 )
2022-03-03 12:53:29 +01:00
mergify[bot]
e169813004
Merge branch 'master' into fix-too-many-columns
2022-03-02 18:46:35 +00:00
Maksim Kita
b1a956c5f1
clang-tidy check performance-move-const-arg fix
2022-03-02 18:15:27 +00:00
Maksim Kita
1f5837359e
clang-tidy check performance-noexcept-move-constructor fix
2022-03-02 18:15:27 +00:00
mreddy017
f893002b69
Fix vulnerable code related to std::move and noexcept
...
This commit fixes the vulnerable code related to std::move and noexcept identified by clangtidy tool.
2022-03-02 18:15:27 +00:00
kssenii
d5952109fb
Merge master
2022-03-02 18:15:25 +01:00
mergify[bot]
add225c83e
Merge branch 'master' into fix-filelog-assertion
2022-03-02 17:06:08 +00:00
Amos Bird
d4cdf04683
Add missing locks ( #34025 )
2022-03-02 16:23:29 +01:00
kssenii
5e84c75942
Fix
2022-03-02 15:31:34 +01:00
kssenii
ef344a581b
Fix bug in FileLog storage
2022-03-02 15:28:17 +01:00
alesapin
b1f5805647
Merge pull request #34609 from ClickHouse/unrestricted-zk-reads
...
allow unrestricted reads from zookeeper
2022-03-02 14:53:12 +01:00
Azat Khuzhin
57f636a1e8
Fix race between INSERT async_insert=1 and system.asynchronous_inserts
...
CI report [1]:
[c190f600f8c6] 2022.03.02 01:07:34.553012 [ 23552 ] {76b6113b-1479-46c9-90ab-e78a3c9f3dbb} executeQuery: Code: 60. DB::Exception: Both table name and UUID are empty. (UNKNOWN_TABLE) (version 22.3.1.1) (from [::1]:42040) (comment: '02015_async_inserts_stress_long.sh') (in query: SELECT * FROM system.asynchronous_inserts FORMAT Null), Stack trace (when copying this message, always include the lines below):
0. ClickHouse/contrib/libcxx/include/exception:133: Poco::Exception::Exception(std::__1::basic_string, std::__1::allocator > const&, int) @ 0xf50e04c in /fasttest-workspace/build/programs/clickhouse
1. ClickHouse/src/Common/Exception.cpp:58: DB::Exception::Exception(std::__1::basic_string, std::__1::allocator > const&, int, bool) @ 0x663ebfa in /fasttest-workspace/build/programs/clickhouse
2. DB::StorageID::assertNotEmpty() const @ 0xbc08591 in /fasttest-workspace/build/programs/clickhouse
3. ClickHouse/contrib/libcxx/include/string:1444: DB::StorageID::getDatabaseName() const @ 0xe50d2b6 in /fasttest-workspace/build/programs/clickhouse
4. ClickHouse/contrib/libcxx/include/string:1957: DB::StorageSystemAsynchronousInserts::fillData(std::__1::vector::mutable_ptr, std::__1::allocator::mutable_ptr > >&, std::__1::shared_ptr, DB::SelectQueryInfo const&) const @ 0xdac636c in /fasttest-workspace/build/programs/clickhouse
[1]: https://s3.amazonaws.com/clickhouse-test-reports/34973/e6fc6a22d5c018961c18247242dd3a40b8c54ff2/fast_test__actions_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-02 15:28:06 +03:00
alesapin
9249c5d50e
Use tryget instead of get
2022-03-02 13:09:12 +03:00
Maksim Kita
53116faeeb
Update MergeTreeIndexFullText.cpp
2022-03-02 11:08:35 +01:00
Kseniia Sumarokova
a9ab149b31
Merge pull request #34859 from Vxider/windowview-multi-column-groupby
...
Fix bugs for multiple columns group by in WindowView
2022-03-02 10:09:47 +01:00
Filatenkov Artur
f48f35cad0
Merge pull request #34975 from Vector-Similarity-Search-for-ClickHouse/fix-typo
...
Fix typo
2022-03-02 09:59:06 +03:00
Anton Popov
d7cd9aa69b
fix reading of missed subcolumns
2022-03-02 03:31:40 +03:00
NikitaEvs
06f47673f4
Fix typo
2022-03-01 21:42:27 +00:00
alesapin
e2989c2b85
Fix storage system zookeeper
2022-03-01 21:23:26 +01:00
alesapin
ec3e4251e1
Fix style
2022-03-01 20:34:25 +01:00
kssenii
a594f388a4
Merge master
2022-03-01 19:43:45 +01:00
kssenii
aa1c71a877
Merge master
2022-03-01 19:25:04 +01:00
kssenii
755e63ed03
Keep compatibility
2022-03-01 19:21:59 +01:00
Anton Popov
c1fdcf7a64
Merge remote-tracking branch 'upstream/master' into HEAD
2022-03-01 20:21:39 +03:00
Anton Popov
04a3a10148
minor fixes
2022-03-01 20:20:53 +03:00
alesapin
cba5fe44a9
Merge branch 'master' into unrestricted-zk-reads
2022-03-01 18:09:21 +01:00
Anton Popov
2758db5341
add more comments
2022-03-01 19:32:55 +03:00
taiyang-li
74bc819f12
fix conflict with master and update contrib/arrow version
2022-03-01 20:14:23 +08:00
kssenii
092ec45b47
Merge master
2022-03-01 12:06:56 +01:00
lgbo-ustc
ca470e1b94
lazy initialization about getting hive metadata in HiveStorage
2022-03-01 19:04:44 +08:00
Kseniia Sumarokova
781621eefe
Merge pull request #34946 from bigo-sg/hive_table_function
...
Add hive table function
2022-03-01 11:28:36 +01:00
alesapin
4b61e4795c
Merge pull request #34949 from nikitamikhaylov/system_log_tables_and_settings
...
Recreate system.{*}_log table on settings changes
2022-03-01 11:15:19 +01:00
lgbo-ustc
5ed41bda9b
fixed code style
2022-03-01 17:20:32 +08:00
lgbo-ustc
5ae99df87c
fxied code style
2022-03-01 15:35:57 +08:00
lgbo-ustc
6e568c1530
update codes
2022-03-01 15:24:40 +08:00
lgbo-ustc
91a45d799e
optimization for first time to read a random access readbuffer
2022-03-01 15:22:07 +08:00
Nikita Mikhaylov
d6036f6da3
Better
...
(cherry picked from commit 4ae445c9e227581ea9f1cbe9aa9d1ba82e1236c9)
2022-02-28 15:27:52 +00:00
Kruglov Pavel
011813957d
Merge pull request #34938 from azat/create-as-ignore-ttl
...
Ignore per-column TTL in CREATE TABLE AS if new table engine does not support it
2022-02-28 16:58:15 +03:00
kssenii
9b64a8fe39
Fix odbc bridge
2022-02-28 14:29:05 +01:00
lgbo-ustc
99cd25d70e
add new table function: hive()
2022-02-28 20:51:33 +08:00
mergify[bot]
b4574a777f
Merge branch 'master' into mvcc_prototype
2022-02-28 11:24:39 +00:00
lgbo-ustc
6473767c99
fixed code style
2022-02-28 17:10:56 +08:00
lgbo-ustc
5885cfd869
fixed bug : unexpected result when using in clause for filtering partitions
2022-02-28 16:47:50 +08:00
Hongbin
c9bc442114
fix comments
2022-02-28 16:44:35 +08:00
Azat Khuzhin
644f9168fa
Ignore per-column TTL in CREATE TABLE AS if new table engine does not support it
...
Follow-up for: #6968
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-28 10:29:26 +03:00
lgbo-ustc
c5e02be44e
fixed code-style
2022-02-28 15:22:54 +08:00
lgbo-ustc
2176d74cd1
Use connection pool in HiveMetastoreClient
...
1. remove lock for hive metastore client access
2. auo reconnect when connection is broken
2022-02-28 15:11:38 +08:00
Hongbin
99bd56e2de
Fix some code comments style
2022-02-28 08:15:37 +08:00
mergify[bot]
8d84d22618
Merge branch 'master' into windowview-multi-column-groupby
2022-02-26 00:50:49 +00:00
kssenii
2ba9010a34
Fix
2022-02-25 17:53:19 +01:00
kssenii
6c8401bfbd
Fix
2022-02-25 16:35:37 +01:00
Lars Eidnes
2629614dfe
Allow LowCardinality strings for ngrambf_v1/tokenbf_v1 indexes. Fixes #21865
2022-02-25 15:36:36 +01:00
Anton Popov
fcdebea925
Merge remote-tracking branch 'upstream/master' into HEAD
2022-02-25 13:41:30 +03:00
kssenii
cfad79bf74
Remove redundant
2022-02-25 09:37:48 +01:00
Alexander Tokmakov
11ae0d144b
fix
2022-02-25 00:51:21 +03:00
Sergei Trifonov
2d25c79e37
analyze select queries from system.zookeeper table with LIKE pattern and fetch nodes using prefix recursively
2022-02-24 13:40:47 +03:00
Vxider
06469eb793
remove blank row
2022-02-24 14:16:24 +08:00
Vxider
43475f79bf
windowview_multi_column_groupby
2022-02-24 14:06:37 +08:00
Alexander Tokmakov
711aad6953
fix
2022-02-24 01:31:21 +03:00
Alexander Tokmakov
aa6b9a2abc
Merge branch 'master' into mvcc_prototype
2022-02-23 23:22:03 +03:00
kssenii
003b807b00
Fix
2022-02-23 20:33:05 +01:00
tavplubix
43626b3ffd
Update src/Storages/FileLog/StorageFileLog.cpp
...
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
2022-02-23 21:07:37 +03:00
Alexander Tokmakov
5a26f856d9
remove trash that shouldn't have been merged
2022-02-22 23:41:33 +03:00
Dmitry Novik
2fd4baaa64
Merge pull request #34387 from nvartolomei/nv/move-part-settings-cleanup
...
Remove useless setting experimental_query_deduplication_send_all_part_uuids
2022-02-22 06:11:00 -08:00
Sergei Trifonov
7fe3bef866
add test for unrestricted zk reads
2022-02-22 16:51:30 +03:00
kssenii
c637385dd0
Merge master
2022-02-22 13:17:51 +01:00
Kseniia Sumarokova
eeea322556
Merge pull request #34629 from amosbird/remotefsimprove
...
Some refactoring and improvement over async and remote buffer related stuff
2022-02-22 11:36:40 +01:00
Dmitry Novik
1df43a7f57
Merge pull request #34385 from nvartolomei/nv/move-part-count
...
Disable optimize_trivial_count when deduplication for part movement feature is enabled
2022-02-21 08:53:09 -08:00
Anton Popov
065305ab65
Merge pull request #34764 from ucasfl/hints-index
...
Add name hints for data skipping indices
2022-02-21 16:50:59 +03:00
Mikhail f. Shiryaev
5ac8cdbc69
Merge pull request #34786 from ClickHouse/make_drop_column_metadata_only
...
Make drop of alias column metadata only
2022-02-21 14:11:55 +01:00
mergify[bot]
314ab73b11
Merge branch 'master' into nv/move-part-settings-cleanup
2022-02-21 10:18:44 +00:00
Dmitry Novik
4428e7aa1b
Merge branch 'master' into nv/move-part-count
2022-02-21 02:14:23 -08:00
alesapin
d7cae5ffb4
Fix build
2022-02-21 11:54:52 +03:00
alesapin
852757219f
Make drop of alias column metadata only
2022-02-21 11:46:16 +03:00
Vitaly Baranov
aee67a6693
Merge pull request #31484 from eungenue/Implement-SSL-X509-certificate-authentication
...
Implement ssl x509 certificate authentication
2022-02-21 11:30:52 +03:00
Vitaly Baranov
0d377de5f0
Support syntax CREATE USER IDENTIFIED WITH ssl_certificate CN ...
2022-02-21 07:01:00 +03:00
Vitaly Baranov
7b97c986cb
Revert "Allow restrictive row policies without permissive"
2022-02-21 06:54:28 +03:00
feng lv
07280e0ab1
Add name hints for data skipping indices
...
fix test
2022-02-20 11:48:22 +00:00
Vitaly Baranov
874b2c8dcb
Merge pull request #34596 from vitlibar/allow-restrictive-without-permissive
...
Allow restrictive row policies without permissive
2022-02-19 21:45:28 +07:00
Azat Khuzhin
fef5f146e7
Fix ENOENT with fsync_part_directory and Vertical merge
...
fsync of the temporary part directory is superfluous anyway, and besides
that directory is not exists at that time, that will lead to ENOENT
error:
2022.02.18 17:02:51.634565 [ 35639 ] {} <Error> void DB::MergeTreeBackgroundExecutor<DB::MergeMutateRuntimeQueue>::routine(DB::TaskRuntimeDataPtr) [Queue = DB::MergeMutateRuntimeQueue]: Code: 107. DB::ErrnoException: Cannot open file /var/lib/clickhouse/data/system/text_log/tmp_merge_202202_1864_3192_14/, errno: 2, strerror: No such file or directory. (FILE_DOESNT_EXIST), Stack trace (when copying this message, always include the lines below):
0. DB::Exception::Exception() @ 0xb26ecfa in /usr/lib/debug/.build-id/01/8c328bd4858d67.debug
1. DB::throwFromErrnoWithPath() @ 0xb2700ea in /usr/lib/debug/.build-id/01/8c328bd4858d67.debug
2. DB::LocalDirectorySyncGuard::LocalDirectorySyncGuard() @ 0x14905531 in /usr/lib/debug/.build-id/01/8c328bd4858d67.debug
3. DB::DiskLocal::getDirectorySyncGuard() const @ 0x148af3e3 in /usr/lib/debug/.build-id/01/8c328bd4858d67.debug
4. DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x157bef13 in /usr/lib/debug/.build-id/01/8c328bd4858d67.debug
Note, that IMergeTreeDataPart::renameTo() anyway will have fsync for the
directory.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-19 07:50:59 +03:00
Nikolai Kochetov
e4d5db6161
Merge pull request #34717 from azat/merge-mutate-memory-tracker
...
Fix possible memory_tracker use-after-free (for async s3 writes) for merges/mutations
2022-02-18 19:28:43 +01:00
Vladimir C
9b7d011ee7
Merge pull request #34529 from vdimir/join-nullable-on-pipeline
...
Apply join_use_nulls on types before join
2022-02-18 18:34:44 +01:00
Azat Khuzhin
65e9b4879d
Fix possible memory_tracker use-after-free for merges/mutations
...
There are two possible cases for execution merges/mutations:
1) from background thread
2) from OPTIMIZE TABLE query
1) is pretty simple, it's memory tracking structure is as follow:
current_thread::memory_tracker = level=Thread / description="(for thread)" ==
background_thread_memory_tracker = level=Thread / description="(for thread)"
current_thread::memory_tracker.parent = level=Global / description="(total)"
So as you can see it is pretty simple and MemoryTrackerThreadSwitcher
does not do anything icky for this case.
2) is complex, it's memory tracking structure is as follow:
current_thread::memory_tracker = level=Thread / description="(for thread)"
current_thread::memory_tracker.parent = level=Process / description="(for query)" ==
background_thread_memory_tracker = level=Process / description="(for query)"
Before this patch to track memory (and related things, like sampling,
profiling and so on) for OPTIMIZE TABLE query dirty hacks was done to
do this, since current_thread memory_tracker was of Thread scope, that
does not have any limits.
And so if will change parent for it to Merge/Mutate memory tracker
(which also does not have some of settings) it will not be correctly
tracked.
To address this Merge/Mutate was set as parent not to the
current_thread memory_tracker but to it's parent, since it's scope is
Process with all settings.
But that parent's memory_tracker is the memory_tracker of the
thread_group, and so if you will have nested ThreadPool inside
merge/mutate (this is the case for s3 async writes, which has been
added in #33291 ) you may get use-after-free of memory_tracker.
Consider the following example:
MemoryTrackerThreadSwitcher()
thread_group.memory_tracker.parent = merge_list_entry->memory_tracker
(see also background_thread_memory_tracker above)
CurrentThread::attachTo()
current_thread.memory_tracker.parent = thread_group.memory_tracker
CurrentThread::detachQuery()
current_thread.memory_tracker.parent = thread_group.memory_tracker.parent
# and this is equal to merge_list_entry->memory_tracker
~MemoryTrackerThreadSwitcher()
thread_group.memory_tracker = thread_group.memory_tracker.parent
So after the following we will get incorrect memory_tracker (from the
mege_list_entry) when the next job in that ThreadPool will not have
thread_group, since in this case it will not try to update the
current_thread.memory_tracker.parent and use-after-free will happens.
So to address the (2) issue, settings from the parent memory_tracker
should be copied to the merge_list_entry->memory_tracker, to avoid
playing with parent memory tracker.
Note, that settings from the query (OPTIMIZE TABLE) is not available at
that time, so it cannot be used (instead of parent's memory tracker
settings).
v2: remove memory_tracker.setOrRaiseHardLimit() from settings
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-18 16:23:54 +03:00
Amos Bird
f459e8fc95
Less getMark calls
2022-02-18 19:55:19 +08:00
Alexander Tokmakov
f4a46a13fb
fixes
2022-02-18 00:26:37 +03:00
Anton Popov
0a7895ebb9
add comments and small refactoring
2022-02-17 22:00:25 +03:00
zvonand
90c857c5e3
merge
2022-02-17 18:23:37 +03:00
tavplubix
0f5ee19d0b
Merge pull request #34633 from zhangjmruc/master
...
For ReplatedMergeTree, early break for multiple leaders case when log has been updated by the other leader
2022-02-17 14:01:50 +03:00
Alexander Tokmakov
dae044f86b
Merge branch 'master' into mvcc_prototype
2022-02-17 13:49:37 +03:00
Kruglov Pavel
6dcb766879
Merge pull request #34465 from Avogar/fix-url-globs
...
Improve schema inference with globs in FIle/S3/HDFS/URL engines
2022-02-17 13:33:27 +03:00
Vitaly Baranov
2de6e8e575
Change type of RowPolicyKind: bool -> enum.
2022-02-17 14:18:10 +07:00
Amos Bird
d3bd8b5f93
Cosmetic fix
2022-02-17 14:31:22 +08:00
Amos Bird
ba19c7cf44
Slightly better interface of compressed buffer
2022-02-17 14:31:22 +08:00
Jianmei Zhang
ef0c3b99ff
Merge remote-tracking branch 'upstream/master'
2022-02-17 14:02:27 +08:00
Sergei Trifonov
b6bb479c48
add setting to enable unrestricted reads from zookeeper
2022-02-16 23:03:44 +03:00
Sergei Trifonov
f342c497ef
fix style
2022-02-16 20:22:03 +03:00
Azat Khuzhin
774744a86d
Fix allow_experimental_projection_optimization with enable_global_with_statement
...
allow_experimental_projection_optimization requires one more
InterpreterSelectQuery, which with enable_global_with_statement will
apply ApplyWithAliasVisitor if the query is not subquery.
But this should not be done for queries from
MergeTreeData::getQueryProcessingStage()/getQueryProcessingStageWithAggregateProjections()
since this will duplicate WITH statements over and over.
This will also fix scalar.xml perf tests, that leads to the following
error now:
scalar.query0.prewarm0: DB::Exception: Stack size too large.
And since it has very long query in the log, this leads to the following
perf test error:
_csv.Error: field larger than field limit (131072)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-16 19:14:47 +03:00
kssenii
47f94120da
Merge master
2022-02-16 14:43:28 +01:00
kssenii
3bd3e51aa0
Fix tests
2022-02-16 14:08:41 +01:00
Mikhail f. Shiryaev
4f84406136
Merge pull request #34641 from ClickHouse/version-and-release
...
refactor version_helper, create release script
2022-02-16 14:00:55 +01:00
Maksim Kita
d6e88f56cd
Merge pull request #34623 from CurtizJ/minor-subcolumns-fix
...
Fix quadratic complexity while adding subcolumns
2022-02-16 12:38:00 +01:00
Mikhail f. Shiryaev
c5db40f679
Deprecate sh script for StorageSystemContributors, update generated file
2022-02-16 12:16:43 +01:00
Nikolai Kochetov
f9d2dae88e
Merge pull request #34424 from yakov-olkhovskiy/ephemeral-column
...
Ephemeral column issue #9436
2022-02-16 12:04:57 +01:00
Kruglov Pavel
dd863ca2a0
Merge branch 'master' into fix-url-globs
2022-02-16 12:45:31 +03:00
Jianmei Zhang
25c761b3b6
Early break for multiple leaders case when log updated by other leader
2022-02-16 16:06:41 +08:00
taiyang-li
a4baec6d26
fix building
2022-02-16 15:12:43 +08:00
taiyang-li
afcb295273
fix compile error
2022-02-16 14:51:56 +08:00
taiyang-li
f19f0d847f
fix code style
2022-02-16 12:23:06 +08:00
taiyang-li
dec083ab44
add minmax index for hivengine; remove libhdfspp dependency for hive engine
2022-02-16 12:19:16 +08:00
Anton Popov
e4fddaa03a
fix quadratic complexity while adding subcolumns
2022-02-16 02:42:50 +03:00
Anton Popov
a661eaf39f
better performance of getting storage snapshot
2022-02-16 02:17:22 +03:00
alesapin
bc2d0ee7c7
Merge pull request #34215 from ClickHouse/revert-34211-revert-34153-add_func_tests_over_s3
...
Add func tests run with s3 and fix several bugs
2022-02-15 19:07:11 +03:00
Alexander Tokmakov
e37ef4560c
fix
2022-02-15 18:00:45 +03:00
Sergei Trifonov
a507f83d8d
allow unrestricted reads from zookeeper
2022-02-15 17:12:37 +03:00
Nikolai Kochetov
ab288642f6
Merge branch 'master' into ephemeral-column
2022-02-15 10:03:34 +00:00
Nikolai Kochetov
d6cbac1ed3
Merge pull request #34577 from ClickHouse/alwasy-remove-unused-actions-for-add-missing-defaults
...
Always remove unused actions from addMissingDefaults
2022-02-15 11:01:29 +01:00
alesapin
447cd56cb9
Fix comments
2022-02-15 12:11:50 +03:00
李扬
f52b67b939
Merge branch 'master' into rocksdb_metacache
2022-02-15 02:16:29 -06:00
Alexander Tokmakov
1e4e569151
Merge branch 'master' into mvcc_prototype
2022-02-15 02:26:47 +03:00
Alexander Tokmakov
ae5aa8c12d
write part version before other files
2022-02-15 02:24:51 +03:00
Alexander Tokmakov
cbd3b45646
add EXPLAIN CURRENT TRANSACTION
2022-02-14 22:47:17 +03:00
alesapin
e15396d90c
Fix race condition:
2022-02-14 22:19:49 +03:00
Nikolai Kochetov
b3ea360cd2
Fix a little bit more
2022-02-14 19:05:30 +00:00
Kseniia Sumarokova
382b8e0388
Merge pull request #34432 from ClickHouse/static-files-disk-uploader-create-symlinks
...
`static-files-disk-uploader`: add a mode to create symlinks
2022-02-14 18:10:53 +01:00
vdimir
99ca89c0ca
Fix StorageJoin and Asof or join_use_nulls in pipeline
2022-02-14 14:14:27 +00:00
alesapin
bb69455395
Merge pull request #34504 from CurtizJ/ttl-move-if-exists
...
Support `TTL TO [DISK|VOLUME] [IF EXISTS]`
2022-02-14 14:56:18 +03:00
alesapin
b75d551281
Fix clang tidy and add check for master
2022-02-14 14:37:41 +03:00
alesapin
b2886a429b
Fix lock during fetch
2022-02-14 12:20:27 +03:00
alesapin
beb4400978
Fix 'same local part' check
2022-02-13 23:08:29 +03:00
alesapin
89373155fc
Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3
2022-02-13 21:07:54 +03:00
Yakov Olkhovskiy
579fe6c97a
major rework, transform added to the insert pipe
2022-02-13 17:42:59 +00:00
zvonand
dcc0f53f8e
updated Interval definitions
2022-02-13 17:54:03 +03:00
Maksim Kita
cb09aadc72
Merge pull request #34524 from FrankChen021/urlengine
...
Fix compression support in URL engine
2022-02-13 14:39:36 +01:00
Maksim Kita
ae1da31d19
Merge pull request #34558 from ucasfl/storage-log
...
try fix data race in StorageLog
2022-02-13 14:33:36 +01:00
mergify[bot]
8a47ed2b3f
Merge branch 'master' into static-files-disk-uploader-create-symlinks
2022-02-13 06:12:39 +00:00
alexey-milovidov
1774836d4f
Merge pull request #34555 from kitaisreal/table-functions-insert-partition-by-refactoring
...
TableFunctionFile added performance test
2022-02-13 09:08:51 +03:00
feng lv
4ccd1c3278
try fix data race in StorageLog
2022-02-13 04:52:16 +00:00
Maksim Kita
e2c8ba9ab2
Added performance test
2022-02-12 16:05:35 +00:00
李扬
daa27d0bda
Merge branch 'master' into rocksdb_metacache
2022-02-12 07:50:12 -06:00
alexey-milovidov
747b6b2058
Merge pull request #34528 from CurtizJ/key-condition-float
...
Fix comparison between integers and floats in index analysis
2022-02-12 10:19:09 +03:00
alexey-milovidov
ea71dc9d11
Merge pull request #34510 from kitaisreal/table-functions-insert-partition-by-refactoring
...
Improve performance of insert into table functions URL, S3, File, HDFS
2022-02-12 10:14:00 +03:00
alexey-milovidov
9bb2eba281
Merge pull request #34429 from ClickHouse/filimonov-SystemAsynchronousInserts
...
asynchronous_inserts engine AsynchronousInserts -> SystemAsynchronousInserts
2022-02-12 10:08:04 +03:00
Frank Chen
7b7bc8f6c3
Address review comment
2022-02-12 10:46:01 +08:00
Dmitry Novik
50131fd732
Fix cancelation for S3 and HDFS
2022-02-11 18:14:55 +00:00
Anton Popov
6a8e35930f
fix comparison with integers and floats in index analysis
2022-02-11 18:20:37 +03:00
tavplubix
6d1fae834a
Merge pull request #34187 from qoega/default-table-engine
...
Default table engine
2022-02-11 18:14:29 +03:00
Anton Popov
2fcd69baf7
fix comparison with integers and floats in index analysis
2022-02-11 17:15:27 +03:00
Alexander Tokmakov
07e66e690d
Merge branch 'master' into mvcc_prototype
2022-02-11 15:53:32 +03:00
mergify[bot]
f099048dd1
Merge branch 'master' into urlengine
2022-02-11 11:03:21 +00:00
Frank Chen
c8cddd7077
Fix compression for URL engine
2022-02-11 17:45:49 +08:00
Anton Popov
f012871a7c
better caching of common types of object columns
2022-02-11 01:20:30 +03:00
Maksim Kita
13cbf79ecb
Improve performance of insert into table functions URL, S3, File, HDFS
2022-02-10 20:06:23 +00:00
alesapin
705529ca03
Followup
2022-02-10 22:50:15 +03:00
alesapin
ef61c9b47c
fix
2022-02-10 22:49:33 +03:00
alesapin
3af06b23f8
POC
2022-02-10 22:45:52 +03:00
alesapin
088f8cacf3
Small improvements
2022-02-10 19:50:21 +03:00
Anton Popov
70986a70a1
support TTL TO [DISK|VOLUME] [IF EXISTS]
2022-02-10 19:26:23 +03:00
Kruglov Pavel
bf8c028533
Merge pull request #34479 from Avogar/fix-filesystem-error
...
Fix possible error 'file_size: Operation not supported'
2022-02-10 14:17:05 +03:00
alesapin
f764da35ca
Also zero copy mutations
2022-02-10 14:15:08 +03:00
Kruglov Pavel
a4f5610764
Merge pull request #34476 from CurtizJ/avoid-settings-copy
...
Avoid unnecessary copying of `Settings`
2022-02-10 14:13:46 +03:00
alesapin
70221b272b
Better solution
2022-02-10 12:57:11 +03:00
Anton Popov
298838f891
avoid unnecessary copying of Settings
2022-02-10 12:13:51 +03:00
avogar
bfa96463ca
Fix possible error 'file_size: Operation not supported'
2022-02-10 09:23:27 +03:00
Anton Popov
dcd7312d75
cache common type on objects in MergeTree
2022-02-09 23:47:53 +03:00
Anton Popov
18940b8637
Merge remote-tracking branch 'upstream/master' into HEAD
2022-02-09 23:38:38 +03:00
alesapin
c587160308
Bad fix which will affect zk and should be changed
2022-02-09 23:06:44 +03:00
alesapin
57037465f5
Trying to fix tests blindly
2022-02-09 22:56:22 +03:00
Kruglov Pavel
538830931b
Merge branch 'master' into fix-url-globs
2022-02-09 20:09:56 +03:00
mergify[bot]
d55eadc3c9
Merge branch 'master' into default-table-engine
2022-02-09 17:08:31 +00:00
Kruglov Pavel
4ec8da73c4
Merge pull request #34448 from Avogar/fix-url-engine
...
Fix bug in URL engine
2022-02-09 19:19:05 +03:00
avogar
84087c1554
Improve schema inference with globs
2022-02-09 19:16:15 +03:00
Yakov Olkhovskiy
a134ab282b
style fixed and minor optimization
2022-02-09 13:51:12 +00:00
Anton Popov
822b58247a
Merge pull request #34132 from CurtizJ/fix-distributed-inserts
...
Fix inserts to distributed tables in case of change of native protocol
2022-02-09 14:58:44 +03:00
Kruglov Pavel
e0e36c256e
Fix style
2022-02-09 14:44:46 +03:00
alesapin
10c3e6e546
Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3
2022-02-09 14:14:58 +03:00
alesapin
72863cc4c3
Fix error
2022-02-09 13:57:10 +03:00
avogar
7baf23f35f
Fix bug URL engine
2022-02-09 13:53:54 +03:00
alesapin
437940b29d
Merge pull request #34422 from ClickHouse/add_settings_to_part_size_in_s3
...
Add settings for multipart upload part size in s3
2022-02-09 12:51:20 +03:00
Kseniia Sumarokova
86956ca08b
Merge pull request #34392 from kssenii/http-buffer-skip-not-found-url-for-globs
...
Allow to skip not found urls for globs
2022-02-09 09:32:09 +01:00
taiyang-li
d04ccc0489
Merge branch 'master' into rocksdb_metacache
2022-02-09 11:54:10 +08:00
Yakov Olkhovskiy
b9777be271
insertion with ephemerals requires columns list, insertion without columns list omits ephemeral
2022-02-09 01:11:29 +00:00
Alexey Milovidov
742620209a
static-files-disk-uploader: add mode to create symlinks
2022-02-09 03:50:50 +03:00
Anton Popov
587d7399ba
support dynamic subcolumns for Memory engine
2022-02-09 03:18:53 +03:00
kssenii
e8a8f46864
Fix style check
2022-02-08 21:58:03 +01:00
alesapin
36909a986f
Fix bug with files remove
2022-02-08 22:21:16 +03:00
filimonov
8d310e4ac3
AsynchronousInserts -> SystemAsynchronousInserts
2022-02-08 19:23:40 +01:00
alesapin
02a93cb852
Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3
2022-02-08 19:42:27 +03:00
alesapin
f6e7cea1f8
Add settings for S3 multipart upload parts size
2022-02-08 19:38:04 +03:00
kssenii
9f2628b8f2
Fix
2022-02-08 16:41:55 +01:00
Kruglov Pavel
cae1517693
Fix build
2022-02-08 17:31:02 +03:00
Nikolai Kochetov
0f7c0c72bd
Merge pull request #34305 from amosbird/projection-fix27
...
Fix various issues when projection is enabled by default
2022-02-08 17:19:56 +03:00
Kruglov Pavel
720310fcf1
Merge branch 'master' into http-buffer-skip-not-found-url-for-globs
2022-02-08 16:56:22 +03:00
Kruglov Pavel
3e216adc17
Merge pull request #34405 from Avogar/fix-segfault-in-url
...
Fix segfault in schema inference from url
2022-02-08 16:48:37 +03:00
Anton Popov
7e9770dcf0
minor enhancements
2022-02-08 15:57:23 +03:00
kssenii
dc5f035265
Fix
2022-02-08 13:28:10 +01:00
Nicolae Vartolomei
50ee264223
Disable projects when allow_experimental_query_deduplication is in use
2022-02-08 12:16:10 +00:00
alesapin
8e9ccbd077
Merge pull request #33933 from sunny19930321/fix-substr-zk-metadata
...
Better local metadata comparison with ZooKeeper metadata
2022-02-08 14:58:46 +03:00
alesapin
b47b0eb1dc
Revert accident change
2022-02-08 14:05:01 +03:00
avogar
e118c89bb2
Fix segfault in schema inference from url
2022-02-08 13:40:03 +03:00
Yatsishin Ilya
bcf4303286
Merge remote-tracking branch 'origin' into default-table-engine
2022-02-08 10:35:29 +00:00
kssenii
3834bdbae0
Fixes
2022-02-08 11:11:18 +01:00
zvonand
0633174b10
Merge branch 'master' of github.com:ClickHouse/ClickHouse into issue_33147
2022-02-08 11:42:31 +03:00
alesapin
3af6012cb4
Revert "Revert "Revert "Revert "Merge pull request #34219 from ClickHouse/revert-34212-revert-33291-add-pool-to-s3-write-buffer""""
...
This reverts commit 2bc2ea485e
2022-02-08 11:01:26 +03:00
alesapin
2bc2ea485e
Revert "Revert "Revert "Merge pull request #34219 from ClickHouse/revert-34212-revert-33291-add-pool-to-s3-write-buffer"""
...
This reverts commit fb77d7a7d5
2022-02-08 10:56:29 +03:00
taiyang-li
b6132d490f
merge master and solve conflict
2022-02-08 15:24:59 +08:00
alexey-milovidov
43ee8ddb5b
Merge pull request #34376 from CurtizJ/fix-reading-empty-arrays
...
Fix consecutive backward seeks in seekable read buffers
2022-02-08 02:27:57 +03:00
Yakov Olkhovskiy
4d5fb56c29
added EPHEMERAL default for column (preliminary)
2022-02-07 23:21:10 +00:00
kssenii
eba3011ada
Fix
2022-02-07 20:40:47 +01:00
Nicolae Vartolomei
7d77678a9f
Remove useless setting experimental_query_deduplication_send_all_part_uuids
...
This setting made sense for testing deduplication before part movement was actually implemented.
allow_experimental_query_deduplication setting is enough and code is covered by test_part_moves_between_shards
2022-02-07 19:03:20 +00:00
zvonand
9456d89fad
Move current work from old laptop
...
skip-checks: true
2022-02-07 21:44:14 +03:00
Anton Popov
1b16db72c3
fix consecutive backward seeks in seekable read buffers
2022-02-07 17:20:26 +03:00
alesapin
523d1059ed
Update KeyDescription.cpp
2022-02-07 13:59:03 +03:00
alesapin
ba28c2c013
Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3
2022-02-07 12:44:56 +03:00
alesapin
fb77d7a7d5
Revert "Revert "Merge pull request #34219 from ClickHouse/revert-34212-revert-33291-add-pool-to-s3-write-buffer""
...
This reverts commit 875e5413ad
2022-02-07 12:36:54 +03:00
alesapin
bb361e3b01
Fix bug with existing part
2022-02-07 12:33:37 +03:00
Yatsishin Ilya
e0803064e6
Merge remote-tracking branch 'origin' into default-table-engine
2022-02-07 08:45:32 +00:00
Amos Bird
2debfc922d
Better projection format and test fixes
2022-02-07 10:47:11 +08:00
alexey-milovidov
fb9ed9acad
Merge pull request #34175 from kitaisreal/bitset-sort-performance-check
...
bitsetsort peformance check
2022-02-07 05:34:57 +03:00
Alexander Tokmakov
45be75b4db
Merge branch 'master' into mvcc_prototype
2022-02-06 23:36:08 +03:00
feng lv
6325d4d9b0
continue of #34317
...
fix
fix
2022-02-06 08:59:17 +00:00
Amos Bird
52aabf98fe
Revise and add more comments
2022-02-06 16:53:54 +08:00
Amos Bird
1ab773cc90
Fix aggregation_in_order with normal projection
2022-02-06 16:46:12 +08:00
Amos Bird
3fab7af541
Bug fix and improvement of minmax_count_projection
2022-02-06 16:46:11 +08:00
Amos Bird
a0ab7a01f1
Adapt minmax_count_projection with ModuleLegacy
2022-02-06 16:46:11 +08:00
Amos Bird
98857de82b
Disable projection for high-order storages
2022-02-06 16:46:10 +08:00
Amos Bird
27fcefd315
Disable projection when doing parallel replica reading
2022-02-06 16:46:10 +08:00
Amos Bird
7674bc986e
Disable projection when there is JOIN or SAMPLE
2022-02-06 16:46:09 +08:00
Maksim Kita
e626b7b2d3
Fixed tests
2022-02-04 19:52:18 +00:00
Maksim Kita
1cd1fb9658
Fixed tests
2022-02-04 19:52:18 +00:00
