chen9t
5cb2301e39
Clean dirty meta cache when part is broken during part loading phase
2022-03-30 20:40:20 +08:00
Antonio Andelic
a3ad99be03
Create CallbackRunner in Storage
2022-03-30 11:24:50 +00:00
kssenii
98ad3f4a91
Fix tests, rename some remote mentions
2022-03-30 12:21:59 +02:00
lgbo-ustc
08e1fdc6b8
Fixed :a elimate AMBIGUOUS_COLUMN_NAME exception
...
elimate AMBIGUOUS_COLUMN_NAME exception in hdfs/s3 cluster when run a query with join
2022-03-30 17:06:38 +08:00
Antonio Andelic
1173eab15e
Use same IOThreadPool for URL and S3
2022-03-30 08:15:20 +00:00
taiyang-li
16bb4c4ad0
respect remote_url_allow_hosts for hive
2022-03-30 15:33:59 +08:00
Antonio Andelic
03a09dc914
Merge pull request #35571 from ClickHouse/parallel-s3-downloading
...
Parallel S3 downloading
2022-03-30 08:14:49 +02:00
Antonio Andelic
d85ed8f2a9
Merge pull request #35655 from ClickHouse/exception-compile-time-message-check
...
Use compile-time check for `Exception` messages
2022-03-30 08:11:32 +02:00
taiyang-li
0af6fdb576
fix building
2022-03-30 11:28:21 +08:00
taiyang-li
fcfabf4a0c
Merge remote-tracking branch 'ck/master' into use_minmax_index
2022-03-30 11:23:24 +08:00
taiyang-li
e76ed3acde
Merge branch 'master' into improve_access_type
2022-03-30 11:04:57 +08:00
taiyang-li
9ffb42d507
Merge branch 'master' into hdfs_config_prefix
2022-03-30 11:02:31 +08:00
Anton Popov
1cba31c305
Merge pull request #35687 from CurtizJ/dynamic-columns-7
...
Fix reading from type object
2022-03-30 00:21:32 +02:00
Alexander Tokmakov
287d858fda
Merge branch 'master' into mvcc_prototype
2022-03-29 16:24:12 +02:00
Dmitry Novik
8f935a72d6
Merge pull request #33230 from CurtizJ/read_in_order_max_rows_to_read
...
Proper handle of 'max_rows_to_read' in case of reading in order of sorting key
2022-03-29 15:16:34 +02:00
Antonio Andelic
ef484547bf
Fix GCC build
2022-03-29 13:16:11 +00:00
taiyang-li
67c3c0be3d
Merge branch 'master' into improve_access_type
2022-03-29 20:42:57 +08:00
taiyang-li
631352ba4b
Merge branch 'master' into hdfs_config_prefix
2022-03-29 20:40:32 +08:00
kssenii
4374415bd7
Fix clang-tidy, style check
2022-03-29 14:20:21 +02:00
Kruglov Pavel
fe60560c03
Merge pull request #35619 from Avogar/check-remote-url-allow-hosts
...
Check remote_url_allow_hosts before schema inference in URL engine
2022-03-29 14:08:02 +02:00
Anton Popov
d42632dc13
Merge pull request #32928 from bigo-sg/rocksdb_metacache
...
Speed up parts loading process of MergeTree by metadata cache
2022-03-29 14:05:22 +02:00
taiyang-li
59727a423e
Merge remote-tracking branch 'ck/master' into use_minmax_index
2022-03-29 19:39:09 +08:00
Maksim Kita
a1a4552740
Merge pull request #35184 from DevTeamBK/clang-tidy-issues
...
clang-tidy report issues with Medium priority
2022-03-29 13:19:54 +02:00
taiyang-li
77314246f3
Merge branch 'master' into hdfs_config_prefix
2022-03-29 18:32:59 +08:00
taiyang-li
834ac3a1b9
Merge remote-tracking branch 'ck/master' into rocksdb_metacache
2022-03-29 18:32:06 +08:00
Nikolai Kochetov
86690f18ee
Merge pull request #35667 from amosbird/projection-fix2
...
Another fix of projection part check
2022-03-29 10:22:01 +02:00
Antonio Andelic
362e877529
Merge branch 'master' into parallel-s3-downloading
2022-03-29 07:05:26 +00:00
Amos Bird
3ab8fc1e7b
Fix projection part check
...
Fix projection part check when projection part type is different from
parent part type.
2022-03-29 10:39:07 +08:00
kssenii
44157e5c80
Merge master
2022-03-29 02:19:32 +02:00
kssenii
c6b1f86306
Merge with master
2022-03-29 02:03:28 +02:00
Nikita Mikhaylov
85282f1b4d
Made schema inference work with s3Cluster table function ( #35544 )
2022-03-29 00:46:35 +02:00
kssenii
1d49a85963
Merge master
2022-03-28 22:51:56 +02:00
kssenii
a2013ebe2b
Better
2022-03-28 22:40:27 +02:00
Kseniia Sumarokova
9a8e9198e5
Merge pull request #35593 from kssenii/named-collections-fix
...
Fix headers with named collections, add compression_method
2022-03-28 22:10:32 +02:00
Alexander Tokmakov
208b242188
Merge branch 'master' into mvcc_prototype
2022-03-28 19:58:06 +02:00
Anton Popov
0e4af89f69
fix reading from type object
2022-03-28 17:23:34 +00:00
mergify[bot]
f09ebea2d8
Merge branch 'master' into read_in_order_max_rows_to_read
2022-03-28 13:30:34 +00:00
taiyang-li
38702be40c
Merge remote-tracking branch 'ck/master' into rocksdb_metacache
2022-03-28 21:03:22 +08:00
mergify[bot]
70b339ea4e
Merge branch 'master' into check-remote-url-allow-hosts
2022-03-28 11:55:48 +00:00
mergify[bot]
a022de4a6e
Merge branch 'master' into fix-materialized-postgresql
2022-03-28 10:55:41 +00:00
Antonio Andelic
d9d826c813
Address PR review
2022-03-28 08:19:23 +00:00
kssenii
c4832f7633
Fix
2022-03-28 09:28:02 +02:00
kssenii
c6706b8bfa
Better
2022-03-25 19:30:13 +01:00
kssenii
141c290344
Fix build
2022-03-25 18:31:15 +01:00
taiyang-li
b79cec6806
Merge branch 'use_minmax_index' of https://github.com/bigo-sg/ClickHouse into use_minmax_index
2022-03-25 23:33:49 +08:00
taiyang-li
eee8949150
fix code
2022-03-25 23:33:46 +08:00
Alexander Tokmakov
9ec0951de5
update comment for mismatching checksums
2022-03-25 15:54:05 +01:00
taiyang-li
4aaa361f2e
Merge remote-tracking branch 'ck/master' into use_minmax_index
2022-03-25 22:48:03 +08:00
taiyang-li
fbadd8fc0e
only deduce absolute path of hdfs config when it's relative path do not exists
2022-03-25 22:28:47 +08:00
taiyang-li
c013cd0b52
Merge branch 'master' into hdfs_config_prefix
2022-03-25 22:13:45 +08:00
taiyang-li
3ca487030d
Merge branch 'master' into improve_access_type
2022-03-25 22:11:08 +08:00
avogar
ecc57b436c
Check remote_url_allow_hosts before schema inference in URL engine
2022-03-25 13:30:45 +00:00
Antonio Andelic
394682b983
Merge branch 'master' into parallel-s3-downloading
2022-03-25 13:00:44 +00:00
kssenii
6769b84638
Merge master
2022-03-24 20:47:43 +01:00
kssenii
d9d9f3bc6d
Fix
2022-03-24 19:10:53 +01:00
kssenii
e91e30aaa4
Fix
2022-03-24 17:10:27 +01:00
Sergei Trifonov
a90e83665d
Merge pull request #35343 from ClickHouse/parallel-multipart-upload-for-s3storage
...
allow parallelization of multipart S3 storage upload; fixes #34350
2022-03-24 15:58:29 +01:00
kssenii
d6ab6de30a
Minor changes
2022-03-24 15:32:08 +01:00
taiyang-li
c0eb9f73a9
remote std::cout
2022-03-24 21:17:43 +08:00
taiyang-li
6eccd09e2b
deduce hdfs config path
2022-03-24 18:04:29 +08:00
Antonio Andelic
62a495a5fc
Small refactor
2022-03-24 09:45:30 +00:00
Nikolai Kochetov
283e20a9a5
Merge pull request #35395 from amosbird/distributedmultiplejoin
...
Validate some thoughts over making sets
2022-03-24 10:30:26 +01:00
Antonio Andelic
c52cee2219
Merge branch 'master' into parallel-s3-downloading
2022-03-24 08:11:52 +00:00
Maksim Kita
1df1721648
Merge pull request #35546 from kitaisreal/refactor-external-models-loader
...
ExternalModelsLoader refactoring
2022-03-24 01:24:45 +01:00
kssenii
d4161b5925
Add optin read_from_cache_if_exists_otherwise_bypass_cache (for merges)
2022-03-23 20:24:00 +01:00
Maksim Kita
3df6a86387
ExternalModelsLoader refactoring
2022-03-23 20:05:17 +01:00
kssenii
d2a3cfe5dc
Cache on all write operations
2022-03-23 19:14:33 +01:00
Yakov Olkhovskiy
1631656803
Merge pull request #34353 from zvonand/issue_33147
...
Extended usage of Milliseconds, Microseconds, Nanoseconds
2022-03-23 13:33:07 -04:00
kssenii
e39aba37a2
Minor changes
2022-03-23 18:11:52 +01:00
Alexander Tokmakov
9aed0507b7
Merge branch 'master' into mvcc_prototype
2022-03-23 18:07:22 +01:00
Sergei Trifonov
bf9c6be6c6
move threadPoolCallbackRunner from Common into Interpreters folder to fix split build issue
2022-03-23 17:57:19 +01:00
Antonio Andelic
cbd20af706
Merge branch 'master' into parallel-s3-downloading
2022-03-23 16:35:05 +00:00
kssenii
c2c7f365ce
Merge master
2022-03-23 16:58:35 +01:00
kssenii
8c1be8950e
Improve test
2022-03-23 16:55:18 +01:00
kssenii
34c0690a69
Better
2022-03-23 15:43:14 +01:00
Antonio Andelic
1a5f5c32bb
Remove commented out code
2022-03-23 14:36:25 +00:00
李扬
9cc528b01f
Update HiveFile.h
2022-03-23 21:57:58 +08:00
xiedeyantu
9441111833
fix INSERT INTO table FROM INFILE does not display progress bar ( #35429 )
...
* fix INSERT INTO table FROM INFILE does not display progress bar
* Update StorageFile.cpp
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-23 14:24:59 +01:00
kssenii
f3e1ca44a9
Add system.remote_data_paths table
2022-03-23 13:01:18 +01:00
Antonio Andelic
93ad209910
Attach memory tracker
2022-03-23 11:52:31 +00:00
Alexander Tokmakov
bcec53aadc
Merge branch 'master' into mvcc_prototype
2022-03-23 11:28:43 +01:00
Antonio Andelic
131b3a091c
Refactor StorageS3
2022-03-23 08:57:01 +00:00
Antonio Andelic
bb6b9f3799
Merge branch 'parallel-downloading-url-engine' into parallel-s3-downloading
2022-03-23 07:46:14 +00:00
Antonio Andelic
9829f2d051
Merge branch 'master' into parallel-downloading-url-engine
2022-03-23 07:41:14 +00:00
taiyang-li
ae3d55c6a2
merge master and fix conflict
2022-03-23 14:31:12 +08:00
taiyang-li
6fe4c3d17e
some minor changes
2022-03-23 12:13:42 +08:00
taiyang-li
8dbf1c60e7
merge master and fix conflict
2022-03-23 11:36:50 +08:00
taiyang-li
68d5b538aa
fix build error
2022-03-23 11:15:42 +08:00
Mikhail f. Shiryaev
e6f5a3f98b
Apply black formatter to all *.py files in the repo
2022-03-22 17:39:58 +01:00
Anton Popov
93dc5c0b81
Merge pull request #35488 from CurtizJ/add-subcolumns-sizes
...
Add sizes of subcolumns to `system.parts_columns` table
2022-03-22 16:53:23 +01:00
Anton Popov
f693eba568
fix tests with approx rows
2022-03-22 14:30:40 +00:00
Antonio Andelic
6785ad165a
Fix issue for mutliple download threads
2022-03-22 14:06:30 +00:00
Antonio Andelic
df99a9cc3c
Merge branch 'parallel-downloading-url-engine' into parallel-s3-downloading
2022-03-22 12:49:35 +00:00
Antonio Andelic
5da3058368
Merge branch 'master' into parallel-downloading-url-engine
2022-03-22 12:20:36 +00:00
Antonio Andelic
246c66bf3e
Merge pull request #35393 from bigo-sg/hive_column_pruning_bug
...
Fix erasing nonexistent column
2022-03-22 08:59:33 +01:00
Anton Popov
0c210a831c
add sizes of subcolumns to system.parts_columns table
2022-03-21 21:12:26 +00:00
kssenii
a9ba14e386
Fix checks
2022-03-21 21:53:46 +01:00
mergify[bot]
9a505476a5
Merge branch 'master' into mvcc_prototype
2022-03-21 20:33:07 +00:00
tavplubix
2fdc937ae1
Merge pull request #35480 from ClickHouse/fix_34586
...
Fix show create for information_schema
2022-03-21 22:20:34 +03:00
Alexander Tokmakov
3c762f566d
Merge branch 'master' into mvcc_prototype
2022-03-21 20:16:29 +01:00
mergify[bot]
9b9959aae0
Merge branch 'master' into read_in_order_max_rows_to_read
2022-03-21 17:33:37 +00:00
Vitaly Baranov
72a2f31984
Merge pull request #33985 from vitlibar/improve-backup-and-restore
...
Improve backup and restore (experimental)
2022-03-21 17:59:51 +01:00
tavplubix
9226ea7b70
Merge pull request #35467 from ClickHouse/resubmit_21474
...
Resubmit #21474
2022-03-21 19:15:28 +03:00
Nikolai Kochetov
5e239762c6
Merge pull request #35334 from amosbird/fixpartitionpruneroverflow
...
Fix incorrect partition pruning when constant in predicate has no supertype of related key columns
2022-03-21 17:05:19 +01:00
Alexander Tokmakov
dba82a7732
fix show create for information_schema
2022-03-21 17:01:39 +01:00
Antonio Andelic
04e6e6ab5e
Add ParallelReadBuffer for S3
2022-03-21 14:52:26 +00:00
kssenii
afd0c64a1a
Better
2022-03-21 15:04:16 +01:00
mergify[bot]
e11ef05c2b
Merge branch 'master' into issue_33147
2022-03-21 13:40:17 +00:00
lgbo-ustc
967d5a8055
Merge remote-tracking branch 'ck/master' into hive_column_pruning_bug
2022-03-21 19:52:06 +08:00
kssenii
eabbce69a7
Add system drop remote filesystem cache query, allow to write cache on insert
2022-03-21 12:32:44 +01:00
Alexander Tokmakov
7cba97aeab
Merge branch 'master' into resubmit_21474
2022-03-21 12:09:00 +01:00
Vitaly Baranov
7a46e7a0d1
Merge branch 'master' into improve-backup-and-restore
2022-03-21 11:49:26 +01:00
Vitaly Baranov
51fc556fcd
Implement BACKUP and RESTORE for MatViews.
2022-03-21 11:45:21 +01:00
mergify[bot]
e5a5ab2a40
Merge branch 'master' into distributedmultiplejoin
2022-03-21 10:00:51 +00:00
Antonio Andelic
b8c43ff2f2
Merge branch 'master' into parallel-downloading-url-engine
2022-03-21 09:11:01 +00:00
kssenii
59b7394caf
WriteSettings
2022-03-21 09:52:48 +01:00
kssenii
1d80262a2a
Add write settings, file cache settings
2022-03-21 09:50:50 +01:00
taiyang-li
49b6f3dfc5
merge master and fix conflict
2022-03-21 15:05:43 +08:00
taiyang-li
bf05b94940
fix build
2022-03-21 15:03:28 +08:00
taiyang-li
d2bea0a72c
Merge branch 'master' into improve_access_type
2022-03-21 14:25:19 +08:00
Alexey Milovidov
0ce4696c49
Revert "[WIP] New row policies"
2022-03-21 08:41:33 +03:00
taiyang-li
7d50bd1eb3
add access type hive
2022-03-21 11:19:45 +08:00
Amos Bird
243de091bb
Validate some thoughts over making sets
2022-03-21 10:58:44 +08:00
lgbo-ustc
f7aa40af5b
update codes
2022-03-21 09:25:20 +08:00
Alexander Tokmakov
9e05b12d2c
Merge branch 'master' into mvcc_prototype
2022-03-20 22:42:26 +01:00
Vitaly Baranov
ce25afb2e9
Storages and databases are hollow by default now.
2022-03-20 20:02:15 +01:00
Vitaly Baranov
c884cf901f
Use BackupEntryFromAppendOnlyFile for Log engine,
...
and use a BackupEntriesBatch for Memort engine to improve performance.
A lot of minor corrections.
2022-03-20 20:02:15 +01:00
Vitaly Baranov
e72a343994
Implement BACKUP & RESTORE for the Memory table engine.
2022-03-20 20:02:15 +01:00
Vitaly Baranov
7f89b98308
Rework BackupSettings and RestoreSettings a little, pass StorageRestoreSettings to storages.
2022-03-20 20:02:15 +01:00
Vitaly Baranov
258a472001
Shorter names: rename IRestoreFromBackupTask -> IRestoreTask.
2022-03-20 20:02:15 +01:00
Vitaly Baranov
5a883c6b5a
Fixed checking access rights while producing backup, add IStorage::hasHollowBackup().
2022-03-20 20:01:54 +01:00
Vitaly Baranov
7a63feb3f7
Make restore tasks explicit.
2022-03-20 20:01:31 +01:00
Vitaly Baranov
9fb50422ce
Merge pull request #35345 from vitlibar/new-row-policies
...
[WIP] New row policies
2022-03-20 20:00:25 +01:00
Alexey Milovidov
9b05801e0f
Merge pull request #35284 from CurtizJ/fix-mutations-sparse-columns
...
Fix mutations in tables with enabled sparse columns
2022-03-20 21:55:59 +03:00
mergify[bot]
7ac606fa65
Merge branch 'master' into parallel-multipart-upload-for-s3storage
2022-03-20 18:25:28 +00:00
alesapin
1bd5736e34
Fix build one more time
2022-03-19 20:00:08 +01:00
alesapin
9e24677a30
Fix build
2022-03-19 18:09:01 +01:00
alesapin
97e84e6dc2
fix build
2022-03-19 17:47:35 +01:00
alesapin
f2c5e2d3a0
Don't spam logs in zero copy replication
2022-03-19 17:31:33 +01:00
Kseniia Sumarokova
a918e68d9a
Merge pull request #35355 from kssenii/add-hosts-filter
...
Add more checks with remoteHostsFilter
2022-03-19 09:41:23 +01:00
Alexander Tokmakov
418d52044a
try fix merge assignment
2022-03-18 20:31:44 +01:00
Alexander Tokmakov
c28008a2df
more comments, minor fixes
2022-03-18 12:01:26 +01:00
Amos Bird
4d85c2fc8b
Use castColumnAccurateOrNull
2022-03-18 17:47:04 +08:00
Amos Bird
10996b1434
Fix mixed constant type during partition pruning
2022-03-18 17:47:03 +08:00
lgbo-ustc
e78cfe3b26
update codes
2022-03-18 15:07:52 +08:00
Alexander Tokmakov
07d952b728
use snapshots for semistructured data, durability fixes
2022-03-17 18:26:18 +01:00
Antonio Andelic
b789a41662
Fix ASAN problems
2022-03-17 15:18:20 +00:00
Anton Popov
11ab24acb1
Merge pull request #35356 from CurtizJ/fix-clang-tidy
...
Fix clang-tidy
2022-03-17 16:17:23 +01:00
Vitaly Baranov
470582e262
Change type of RowPolicyKind: bool -> enum.
2022-03-17 15:28:05 +01:00
Alexander Tokmakov
d04dc03fa4
Merge branch 'master' into mvcc_prototype
2022-03-17 15:24:32 +01:00
Nikolai Kochetov
ee9c2ec735
Merge pull request #34780 from azat/mt-delayed-part-flush
...
Do not delay final part writing by default (fixes possible Memory limit exceeded during INSERT)
2022-03-17 12:30:51 +01:00
Anton Popov
fe2dc85788
fix clang-tidy
2022-03-17 11:16:32 +00:00
kssenii
a2cd165d38
Add remote host filter
2022-03-17 11:48:42 +01:00
alesapin
92ab33f560
Merge pull request #35348 from ClickHouse/fix_yet_another_zero_copy_bug
...
Fix race in zero copy replication
2022-03-17 11:34:38 +01:00
alesapin
bb251938dc
Merge pull request #35344 from ClickHouse/changelog-22.3
...
Changelog 22.3
2022-03-17 11:25:36 +01:00
Antonio Andelic
103a3fa140
Merge branch 'master' into parallel-downloading-url-engine
2022-03-17 10:11:58 +00:00
Antonio Andelic
44de5a581c
Fix split build
2022-03-17 10:09:16 +00:00
Antonio Andelic
cc623bbf5e
Fix redirect_url tests
2022-03-17 08:03:22 +00:00
Alexey Milovidov
68ef49ea51
Fix something stupid
2022-03-17 05:57:13 +01:00
alesapin
03ff6d52f6
Fix race in zero copy replication
2022-03-16 21:51:33 +01:00
Anton Popov
de2cc23e15
fix race
2022-03-16 20:16:59 +00:00
Alexander Tokmakov
4b3e13a4fe
Merge branch 'master' into mvcc_prototype
2022-03-16 21:06:19 +01:00
Alexander Tokmakov
c2ac8d4a5c
review fixes
2022-03-16 21:05:34 +01:00
Sergei Trifonov
bbe657ba24
allow parallelization of multipart S3 storage upload fixes #34350
2022-03-16 19:41:41 +01:00
Antonio Andelic
0979155f2f
Address PR comments
2022-03-16 15:45:17 +00:00
Anton Popov
0ba78c3c3a
Merge remote-tracking branch 'upstream/master' into HEAD
2022-03-16 15:28:09 +00:00
Antonio Andelic
84f86bf3da
Limit number of bytes read per worker
2022-03-16 12:34:14 +00:00
Antonio Andelic
2291b7c1e1
Merge branch 'master' into parallel-downloading-url-engine
2022-03-16 09:59:52 +00:00
Nikolai Kochetov
a380aa6b8a
Merge pull request #35294 from ClickHouse/reload-remote_url_allow_hosts
...
Reload remote_url_allow_hosts after config update.
2022-03-15 22:07:16 +01:00
Alexander Tokmakov
1f571b7734
Merge branch 'master' into mvcc_prototype
2022-03-15 14:45:06 +01:00
Nikolai Kochetov
97aa6c82ce
Reload remote_url_allow_hosts after config update.
2022-03-15 13:00:31 +00:00
Anton Popov
b5a04ff938
Merge pull request #35207 from bigo-sg/add_distributed_depth_in_querylog
...
Add column distributed_depth for system.query_log and system.processes
2022-03-15 13:38:21 +01:00
Kseniia Sumarokova
427ae808ba
Merge pull request #35191 from DevTeamBK/RemoteHostFilter_improvement
...
Added RemoteHostFilter check for MYSQL and postgresSQL
2022-03-15 09:58:08 +01:00
Antonio Andelic
067b79b00b
Merge branch 'master' into parallel-downloading-url-engine
2022-03-15 07:55:41 +00:00
lgbo-ustc
abfaa82bca
fixed hive query bugs
2022-03-15 12:01:34 +08:00
Anton Popov
ccbddd53a3
fix mutations in tables with enabled sparse columns
2022-03-15 01:48:21 +00:00
alesapin
fbb1ebd9b8
Merge pull request #35274 from CurtizJ/fix-check-table-sparse-columns
...
Fix check table in case when there exist sparse columns
2022-03-14 21:56:04 +01:00
Alexander Tokmakov
9702b5177d
Merge branch 'master' into mvcc_prototype
2022-03-14 21:45:38 +01:00
Alexander Tokmakov
278d779a01
log cleanup, more comments
2022-03-14 21:43:34 +01:00
Maksim Kita
2fdcf53a76
Fix clang-tidy warnings in Server, Storages folders
2022-03-14 18:17:35 +00:00
Anton Popov
063917786e
minor fixes
2022-03-14 17:29:18 +00:00
Anton Popov
36ec379aeb
Merge remote-tracking branch 'upstream/master' into HEAD
2022-03-14 16:28:35 +00:00
Anton Popov
428bbd6377
fix check table in case when there exist sparse columns
2022-03-14 15:22:23 +00:00
Heena Bansal
c774458f96
Update src/Storages/StoragePostgreSQL.cpp
...
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
2022-03-14 10:38:12 -04:00
Heena Bansal
01c8b2f71e
Update src/Storages/StorageMySQL.cpp
...
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
2022-03-14 10:37:59 -04:00
Antonio Andelic
d3353f3f0c
Merge branch 'master' into parallel-downloading-url-engine
2022-03-14 12:47:23 +00:00
Antonio Andelic
556fe2bcc5
Don't send RANGE with HEAD call
2022-03-14 12:46:22 +00:00
Antonio Andelic
ed3d71d83f
Merge pull request #35072 from azat/buffer-memory-tracker-leak
...
Avoid MEMORY_LIMIT_EXCEEDED during INSERT into Buffer with AggregateFunction
2022-03-14 12:31:04 +01:00
Maksim Kita
ad6b3693e1
Merge pull request #35123 from zhanghuajieHIT/fix_build_fail_with_gcc
...
fix build fail with gcc
2022-03-14 10:36:15 +01:00
Antonio Andelic
9dda2863d3
Split download threads when multiple URLs are used
2022-03-14 09:27:09 +00:00
HeenaBansal2009
09be30ac26
Added suggestion from review comments
2022-03-11 07:33:23 -08:00
zhanghuajie
53a8987b3b
fix build fail with gcc --fix warnings without disabling some parameters
2022-03-11 21:59:19 +08:00
Antonio Andelic
f5d3a8a31d
Polishing
2022-03-11 13:38:19 +00:00
taiyang-li
1f9e050152
add column distributed_depth for system.query_log and system.processes
2022-03-11 17:57:34 +08:00
Antonio Andelic
28e9508c4e
Improve shared pool and add settings
2022-03-11 08:33:34 +00:00
Antonio Andelic
cce318273b
Use a shared IO thread pool
2022-03-11 08:33:34 +00:00
Antonio Andelic
df0f5e20d0
Refactor
2022-03-11 08:33:34 +00:00
Antonio Andelic
e051587fc2
Format code
2022-03-11 08:33:34 +00:00
Antonio Andelic
29c32ed831
Refactor code
2022-03-11 08:33:34 +00:00
Antonio Andelic
0a1a3a230e
Add support for parallel http range requests
2022-03-11 08:33:34 +00:00
Antonio Andelic
58557b9bec
Test check for ranges support
2022-03-11 08:33:34 +00:00
Kseniia Sumarokova
818459b9f0
Merge pull request #33717 from kssenii/local-cache-for-remote-fs
...
Local cache for remote filesystem
2022-03-11 07:23:10 +01:00
Alexander Tokmakov
7f47f20aba
add kill transaction query
2022-03-10 22:29:58 +01:00
Anton Popov
37efe2ddb5
Apply suggestions from code review
...
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
2022-03-10 22:24:19 +01:00
alesapin
c0d8ccc91b
Merge pull request #35178 from Varinara/master
...
Added disk_name to system.part_log
2022-03-10 22:22:37 +01:00
HeenaBansal2009
d981463b05
Added RemoteHostFilter check for MYSQL and postgresSQL
2022-03-10 08:58:48 -08:00
Maksim Kita
493169910b
Merge pull request #35174 from zhangyifan27/fix_typo
...
fix typos
2022-03-10 17:10:44 +01:00
Kseniia Sumarokova
3fc399b6e9
Merge pull request #35158 from kssenii/fix-materialized-postgresql
...
Fix materialised postrgesql adding new table after manually removing it
2022-03-10 17:02:32 +01:00
Kseniia Sumarokova
e30b0c5d57
Merge pull request #35162 from kssenii/fix-materialized-postgresql-table-override
...
Fix materialised postgres `table overrides` for partition by, etc
2022-03-10 17:01:24 +01:00
Varinara
f5523f7ff0
added disk_name to system.part_log
2022-03-10 18:44:19 +03:00
Rajkumar
3d3b6d1956
clang-tidy report issues with Medium priority
2022-03-10 07:23:49 -08:00
mergify[bot]
df01290e73
Merge branch 'master' into fix_typo
2022-03-10 13:35:04 +00:00
Alexander Tokmakov
061fa6a6f2
Merge branch 'master' into mvcc_prototype
2022-03-10 13:13:04 +01:00
kssenii
1dc3f36a11
Better
2022-03-10 12:19:20 +01:00
kssenii
787a0805a5
Merge master
2022-03-10 11:42:19 +01:00
Kseniia Sumarokova
e6ee891c9c
Merge pull request #34957 from bigo-sg/hive_random_access_file_cache
...
Optimization for first time to read a random access readbuffer in hive
2022-03-10 11:36:22 +01:00
zhangyifan27
e6fa9f699a
fix typo
2022-03-10 18:29:42 +08:00
kssenii
3cd1da1e11
Fix
2022-03-10 11:11:59 +01:00
Alexander Tokmakov
0906b59fba
fixes
2022-03-09 21:38:18 +01:00
kssenii
af9d8d278e
Fix
2022-03-09 19:25:43 +01:00
Vladimir C
ce266b5a3e
Merge pull request #35146 from amosbird/fixpartitionprunerin
2022-03-09 13:23:45 +01:00
Nikolai Kochetov
6bfee7aca2
Merge pull request #35132 from azat/parallel_distributed_insert_select-view
...
Support view() for parallel_distributed_insert_select
2022-03-09 09:10:34 +01:00
Azat Khuzhin
e2960e1a52
Avoid MEMORY_LIMIT_EXCEEDED during INSERT into Buffer with AggregateFunction
...
In case of Buffer table has columns of AggregateFunction type,
aggregate states for such columns will be allocated from the query
context but those states can be destroyed from the server context (in
case of background flush), and thus memory will be leaked from the query
since aggregate states can be shared, and eventually this will lead to
MEMORY_LIMIT_EXCEEDED error.
To avoid this, prohibit sharing the aggregate states.
But note, that this problem only about memory accounting, not memory
usage itself.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-09 10:57:49 +03:00
Amos Bird
a19224bc9b
Fix partition pruner: non-monotonic function IN
2022-03-09 15:48:42 +08:00
Azat Khuzhin
3a5a39a9df
Do not delay final part writing by default
...
For async s3 writes final part flushing was defered until all the INSERT
block was processed, however in case of too many partitions/columns you
may exceed max_memory_usage limit (since each stream has overhead).
Introduce max_insert_delayed_streams_for_parallel_writes (with default
to 1000 for S3, 0 otherwise), to avoid this.
This should "Memory limit exceeded" errors in performance tests.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-08 22:17:36 +03:00
Alexander Tokmakov
d2f838bd91
fix another race condition
2022-03-08 20:11:47 +01:00
Azat Khuzhin
4843e210c3
Support view() for parallel_distributed_insert_select
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-08 22:05:57 +03:00
Azat Khuzhin
ced34dea84
Take flush_time into account for scheduling background flush of the Buffer
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-08 21:58:10 +03:00
kssenii
5260822964
Merge master
2022-03-08 18:21:28 +01:00
kssenii
e231c3a3e0
Fix split build
2022-03-08 18:05:55 +01:00
Azat Khuzhin
c4b6342853
Improvements for parallel_distributed_insert_select (and related) ( #34728 )
...
* Add a warning if parallel_distributed_insert_select was ignored
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
* Respect max_distributed_depth for parallel_distributed_insert_select
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
* Print warning for non applied parallel_distributed_insert_select only for initial query
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
* Remove Cluster::getHashOfAddresses()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
* Forbid parallel_distributed_insert_select for remote()/cluster() with different addresses
Before it uses empty cluster name (getClusterName()) which is not
correct, compare all addresses instead.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
* Fix max_distributed_depth check
max_distributed_depth=1 must mean not more then one distributed query,
not two, since max_distributed_depth=0 means no limit, and
distribute_depth is 0 for the first query.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
* Fix INSERT INTO remote()/cluster() with parallel_distributed_insert_select
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
* Add a test for parallel_distributed_insert_select with cluster()/remote()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
* Return <remote> instead of empty cluster name in Distributed engine
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
* Make user with sharding_key and w/o in remote()/cluster() identical
Before with sharding_key the user was "default", while w/o it it was
empty.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-08 15:24:39 +01:00
Antonio Andelic
bc5d7aea57
Merge pull request #34876 from azat/long-INSERT-fix
...
Fix possible "Part directory doesn't exist" during INSERT
2022-03-08 12:44:53 +01:00
Kseniia Sumarokova
1eb2bae792
Merge pull request #34954 from bigo-sg/hive_read_columns_pruning
...
read columns pruning for hive
2022-03-08 10:17:24 +01:00
lgbo-ustc
256e92ffee
Merge remote-tracking branch 'ck/master' into hive_random_access_file_cache
2022-03-08 14:14:40 +08:00
Azat Khuzhin
caffc144b5
Fix possible "Part directory doesn't exist" during INSERT
...
In #33291 final part commit had been defered, and now it can take
significantly more time, that may lead to "Part directory doesn't exist"
error during INSERT:
2022.02.21 18:18:06.979881 [ 11329 ] {insert} <Debug> executeQuery: (from 127.1:24572, user: default) INSERT INTO db.table (...) VALUES
2022.02.21 20:58:03.933593 [ 11329 ] {insert} <Trace> db.table: Renaming temporary part tmp_insert_20220214_18044_18044_0 to 20220214_270654_270654_0.
2022.02.21 21:16:50.961917 [ 11329 ] {insert} <Trace> db.table: Renaming temporary part tmp_insert_20220214_18197_18197_0 to 20220214_270689_270689_0.
...
2022.02.22 21:16:57.632221 [ 64878 ] {} <Warning> db.table: Removing temporary directory /clickhouse/data/db/table/tmp_insert_20220214_18232_18232_0/
...
2022.02.23 12:23:56.277480 [ 11329 ] {insert} <Trace> db.table: Renaming temporary part tmp_insert_20220214_18232_18232_0 to 20220214_273459_273459_0.
2022.02.23 12:23:56.299218 [ 11329 ] {insert} <Error> executeQuery: Code: 107. DB::Exception: Part directory /clickhouse/data/db/table/tmp_insert_20220214_18232_18232_0/ doesn't exist. Most likely it is a logical error. (FILE_DOESNT_EXIST) (version 22.2.1.1) (from 127.1:24572) (in query: INSERT INTO db.table (...) VALUES), Stack trace (when copying this message, always include the lines below):
Follow-up for: #28760
Refs: #33291
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-08 07:44:11 +03:00
lgbo-ustc
a8cfc2458a
update codes
2022-03-08 11:55:15 +08:00
taiyang-li
b4174b0bef
merge master and fix conflicts
2022-03-08 11:39:25 +08:00
Maksim Kita
2f9361008b
Merge pull request #35089 from 1lann/1lann/fix-update_lag-typo
...
Fix typo of update_lag
2022-03-07 23:12:35 +01:00
Alexander Tokmakov
8acfb8d27f
Merge branch 'master' into mvcc_prototype
2022-03-07 17:40:15 +01:00
Alexander Tokmakov
ea2f65fef6
fix tests with DiskS3
2022-03-07 17:35:47 +01:00
Kseniia Sumarokova
5511f2f6e6
Merge pull request #34940 from bigo-sg/hive_client_connection_pool
...
Use connection pool in HiveMetastoreClient
2022-03-07 17:14:56 +01:00
Kseniia Sumarokova
28b9ec01c0
Merge pull request #34945 from bigo-sg/hive_bug_fixed
...
unexpected result when use `in` in hive query
2022-03-07 17:13:11 +01:00
Anton Popov
0bc57da238
Merge remote-tracking branch 'upstream/master' into HEAD
2022-03-07 14:46:08 +00:00
alesapin
d90bee1df9
Merge pull request #35080 from azat/mutate-exceptions
...
Do not hide exceptions during mutations
2022-03-07 12:22:49 +01:00
Vladimir C
678f05ca1e
Merge pull request #34912 from kssenii/fix-filelog-metadata-path
2022-03-07 11:45:29 +01:00
1lann
5423c5a45c
Fix typo of update_lag
...
In external dictionary providers, the allowed keys for configuration seemed to have a typo
of "update_lag" as "update_tag", preventing the use of "update_lag". This change fixes that.
2022-03-07 18:31:20 +08:00
lgbo-ustc
8ae5296ee8
fixed compile errors
2022-03-07 17:26:48 +08:00
lgbo-ustc
eab925554d
fixed code styles
2022-03-07 14:01:03 +08:00
lgbo-ustc
0c83b96d8c
fixed code style
2022-03-07 13:13:33 +08:00
lgbo-ustc
cfeedd2cb5
fixed code style
2022-03-07 12:28:31 +08:00
lgbo-ustc
4507cc58aa
update codes
2022-03-07 12:05:07 +08:00
lgbo-ustc
c37eedd887
update codes
2022-03-07 10:30:54 +08:00
lgbo-ustc
75a50a30c4
update codes
2022-03-07 09:43:53 +08:00
lgbo-ustc
d907b70cc4
update codes: get actual read block
2022-03-07 09:26:05 +08:00
lgbo-ustc
f4d8fb46c5
update codes
2022-03-07 09:26:05 +08:00
lgbo-ustc
62c1bd5ae9
hive read columns pruning
2022-03-07 09:26:05 +08:00
Azat Khuzhin
bc224dee36
Do not hide exceptions during mutations
...
system.mutations includes only the message, but not stacktrace, and it
is not always obvious to understand the culprit w/o stacktrace.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-06 13:39:49 +03:00
Kseniia Sumarokova
3ec6cd3128
Update StorageFileLog.cpp
2022-03-06 11:03:22 +01:00
alexey-milovidov
f9b7df6ba1
Merge pull request #35050 from CurtizJ/fix-async-inserts-system-table
...
Fix reading from `system.asynchronous_inserts` table
2022-03-06 02:25:53 +03:00
Maksim Kita
7ae1f0fa3b
Merge pull request #34911 from larspars/master
...
Allow LowCardinality strings for ngrambf_v1/tokenbf_v1 indexes. Fixes #21865
2022-03-04 19:17:48 +01:00
Azat Khuzhin
2ef9d32448
Revert "Remove VERSION_DATE from system.build_options"
...
As requested by @kitaisreal
This reverts commit 4a404532fb
2022-03-04 15:32:54 +03:00
Azat Khuzhin
4a404532fb
Remove VERSION_DATE from system.build_options
...
It was set only bu utils/release/release_lib.sh, and seems that this
script is not used anymore, at least that part of it.
Also note, that GIT_DATE is the same, and it is date time, not only
date.
Plus VERSION_DATE is not installed for releases anyway.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-04 15:31:37 +03:00
Azat Khuzhin
b0f964a14f
Remove LIBRARY_ARCHITECTURE from system.build_options
...
CMAKE_LIBRARY_ARCHITECTURE and it is useless, since it is reported only
if the compiler reports subdir arch triplet [1]
[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1531678
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-04 15:31:32 +03:00
Azat Khuzhin
494fe91f86
Fix LINK_FLAGS in system.build_options
...
Fixes: 79f6f5a202
2022-03-04 15:31:32 +03:00
Azat Khuzhin
c426eef07d
Fix generating USE_* for system.build_options
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-04 15:31:32 +03:00
Anton Popov
c836a57000
fix reading from system.asynchronous_inserts table
2022-03-04 11:46:15 +00:00
Anton Popov
df3b07fe7c
Merge remote-tracking branch 'upstream/master' into HEAD
2022-03-03 22:25:28 +00:00
Anton Popov
aea7bfb59a
Merge pull request #34992 from azat/fix-asynchronous_inserts-race
...
Fix race between INSERT async_insert=1 and system.asynchronous_inserts
2022-03-03 20:55:19 +03:00
kssenii
d19f199e93
Revert
2022-03-03 15:25:27 +01:00
Kseniia Sumarokova
b11b34dc8c
Merge pull request #34849 from kssenii/fix-too-many-columns
...
Fix reading too many columns for s3 and url storages
2022-03-03 13:57:22 +01:00
Kseniia Sumarokova
ad09554c4c
Merge pull request #34996 from kssenii/fix-filelog-assertion
...
Fix possible segfault in filelog storage
2022-03-03 13:52:54 +01:00
Frank Chen
b4829465d9
Improve the opentelemetry span logs for INSERT on distributed table ( #34480 )
2022-03-03 12:53:29 +01:00
mergify[bot]
e169813004
Merge branch 'master' into fix-too-many-columns
2022-03-02 18:46:35 +00:00
Maksim Kita
b1a956c5f1
clang-tidy check performance-move-const-arg fix
2022-03-02 18:15:27 +00:00
Maksim Kita
1f5837359e
clang-tidy check performance-noexcept-move-constructor fix
2022-03-02 18:15:27 +00:00
mreddy017
f893002b69
Fix vulnerable code related to std::move and noexcept
...
This commit fixes the vulnerable code related to std::move and noexcept identified by clangtidy tool.
2022-03-02 18:15:27 +00:00
kssenii
d5952109fb
Merge master
2022-03-02 18:15:25 +01:00
mergify[bot]
add225c83e
Merge branch 'master' into fix-filelog-assertion
2022-03-02 17:06:08 +00:00
Amos Bird
d4cdf04683
Add missing locks ( #34025 )
2022-03-02 16:23:29 +01:00
kssenii
5e84c75942
Fix
2022-03-02 15:31:34 +01:00
kssenii
ef344a581b
Fix bug in FileLog storage
2022-03-02 15:28:17 +01:00
alesapin
b1f5805647
Merge pull request #34609 from ClickHouse/unrestricted-zk-reads
...
allow unrestricted reads from zookeeper
2022-03-02 14:53:12 +01:00
Azat Khuzhin
57f636a1e8
Fix race between INSERT async_insert=1 and system.asynchronous_inserts
...
CI report [1]:
[c190f600f8c6] 2022.03.02 01:07:34.553012 [ 23552 ] {76b6113b-1479-46c9-90ab-e78a3c9f3dbb} executeQuery: Code: 60. DB::Exception: Both table name and UUID are empty. (UNKNOWN_TABLE) (version 22.3.1.1) (from [::1]:42040) (comment: '02015_async_inserts_stress_long.sh') (in query: SELECT * FROM system.asynchronous_inserts FORMAT Null), Stack trace (when copying this message, always include the lines below):
0. ClickHouse/contrib/libcxx/include/exception:133: Poco::Exception::Exception(std::__1::basic_string, std::__1::allocator > const&, int) @ 0xf50e04c in /fasttest-workspace/build/programs/clickhouse
1. ClickHouse/src/Common/Exception.cpp:58: DB::Exception::Exception(std::__1::basic_string, std::__1::allocator > const&, int, bool) @ 0x663ebfa in /fasttest-workspace/build/programs/clickhouse
2. DB::StorageID::assertNotEmpty() const @ 0xbc08591 in /fasttest-workspace/build/programs/clickhouse
3. ClickHouse/contrib/libcxx/include/string:1444: DB::StorageID::getDatabaseName() const @ 0xe50d2b6 in /fasttest-workspace/build/programs/clickhouse
4. ClickHouse/contrib/libcxx/include/string:1957: DB::StorageSystemAsynchronousInserts::fillData(std::__1::vector::mutable_ptr, std::__1::allocator::mutable_ptr > >&, std::__1::shared_ptr, DB::SelectQueryInfo const&) const @ 0xdac636c in /fasttest-workspace/build/programs/clickhouse
[1]: https://s3.amazonaws.com/clickhouse-test-reports/34973/e6fc6a22d5c018961c18247242dd3a40b8c54ff2/fast_test__actions_.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-02 15:28:06 +03:00
alesapin
9249c5d50e
Use tryget instead of get
2022-03-02 13:09:12 +03:00
Maksim Kita
53116faeeb
Update MergeTreeIndexFullText.cpp
2022-03-02 11:08:35 +01:00
Kseniia Sumarokova
a9ab149b31
Merge pull request #34859 from Vxider/windowview-multi-column-groupby
...
Fix bugs for multiple columns group by in WindowView
2022-03-02 10:09:47 +01:00
Filatenkov Artur
f48f35cad0
Merge pull request #34975 from Vector-Similarity-Search-for-ClickHouse/fix-typo
...
Fix typo
2022-03-02 09:59:06 +03:00
Anton Popov
d7cd9aa69b
fix reading of missed subcolumns
2022-03-02 03:31:40 +03:00
NikitaEvs
06f47673f4
Fix typo
2022-03-01 21:42:27 +00:00
alesapin
e2989c2b85
Fix storage system zookeeper
2022-03-01 21:23:26 +01:00
alesapin
ec3e4251e1
Fix style
2022-03-01 20:34:25 +01:00
kssenii
a594f388a4
Merge master
2022-03-01 19:43:45 +01:00
kssenii
aa1c71a877
Merge master
2022-03-01 19:25:04 +01:00
kssenii
755e63ed03
Keep compatibility
2022-03-01 19:21:59 +01:00
Anton Popov
c1fdcf7a64
Merge remote-tracking branch 'upstream/master' into HEAD
2022-03-01 20:21:39 +03:00
Anton Popov
04a3a10148
minor fixes
2022-03-01 20:20:53 +03:00
alesapin
cba5fe44a9
Merge branch 'master' into unrestricted-zk-reads
2022-03-01 18:09:21 +01:00
Anton Popov
2758db5341
add more comments
2022-03-01 19:32:55 +03:00
taiyang-li
74bc819f12
fix conflict with master and update contrib/arrow version
2022-03-01 20:14:23 +08:00
kssenii
092ec45b47
Merge master
2022-03-01 12:06:56 +01:00
