thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 19:32:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
84,049 Commits 9,319 Branches 1,783 Tags 2.1 GiB
3cd1da1e11
Commit Graph

7062 Commits

Author SHA1 Message Date
Amos Bird
f459e8fc95
Less getMark calls 2022-02-18 19:55:19 +08:00
tavplubix
0f5ee19d0b
Merge pull request #34633 from zhangjmruc/master 
For ReplatedMergeTree, early break for multiple leaders case when log has been updated by the other leader
 2022-02-17 14:01:50 +03:00
Kruglov Pavel
6dcb766879
Merge pull request #34465 from Avogar/fix-url-globs 
Improve schema inference with globs in FIle/S3/HDFS/URL engines
 2022-02-17 13:33:27 +03:00
Vitaly Baranov
2de6e8e575 Change type of RowPolicyKind: bool -> enum. 2022-02-17 14:18:10 +07:00
Amos Bird
d3bd8b5f93
Cosmetic fix 2022-02-17 14:31:22 +08:00
Amos Bird
ba19c7cf44
Slightly better interface of compressed buffer 2022-02-17 14:31:22 +08:00
Jianmei Zhang
ef0c3b99ff Merge remote-tracking branch 'upstream/master' 2022-02-17 14:02:27 +08:00
Sergei Trifonov
b6bb479c48 add setting to enable unrestricted reads from zookeeper 2022-02-16 23:03:44 +03:00
Sergei Trifonov
f342c497ef fix style 2022-02-16 20:22:03 +03:00
Azat Khuzhin
774744a86d Fix allow_experimental_projection_optimization with enable_global_with_statement 
allow_experimental_projection_optimization requires one more
InterpreterSelectQuery, which with enable_global_with_statement will
apply ApplyWithAliasVisitor if the query is not subquery.

But this should not be done for queries from
MergeTreeData::getQueryProcessingStage()/getQueryProcessingStageWithAggregateProjections()
since this will duplicate WITH statements over and over.

This will also fix scalar.xml perf tests, that leads to the following
error now:

    scalar.query0.prewarm0: DB::Exception: Stack size too large.

And since it has very long query in the log, this leads to the following
perf test error:

    _csv.Error: field larger than field limit (131072)

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-16 19:14:47 +03:00
Mikhail f. Shiryaev
4f84406136
Merge pull request #34641 from ClickHouse/version-and-release 
refactor version_helper, create release script
 2022-02-16 14:00:55 +01:00
Maksim Kita
d6e88f56cd
Merge pull request #34623 from CurtizJ/minor-subcolumns-fix 
Fix quadratic complexity while adding subcolumns
 2022-02-16 12:38:00 +01:00
Mikhail f. Shiryaev
c5db40f679
Deprecate sh script for StorageSystemContributors, update generated file 2022-02-16 12:16:43 +01:00
Nikolai Kochetov
f9d2dae88e
Merge pull request #34424 from yakov-olkhovskiy/ephemeral-column 
Ephemeral column issue #9436
 2022-02-16 12:04:57 +01:00
Kruglov Pavel
dd863ca2a0
Merge branch 'master' into fix-url-globs 2022-02-16 12:45:31 +03:00
Jianmei Zhang
25c761b3b6 Early break for multiple leaders case when log updated by other leader 2022-02-16 16:06:41 +08:00
Anton Popov
e4fddaa03a fix quadratic complexity while adding subcolumns 2022-02-16 02:42:50 +03:00
alesapin
bc2d0ee7c7
Merge pull request #34215 from ClickHouse/revert-34211-revert-34153-add_func_tests_over_s3 
Add func tests run with s3 and fix several bugs
 2022-02-15 19:07:11 +03:00
Sergei Trifonov
a507f83d8d allow unrestricted reads from zookeeper 2022-02-15 17:12:37 +03:00
Nikolai Kochetov
ab288642f6 Merge branch 'master' into ephemeral-column 2022-02-15 10:03:34 +00:00
Nikolai Kochetov
d6cbac1ed3
Merge pull request #34577 from ClickHouse/alwasy-remove-unused-actions-for-add-missing-defaults 
Always remove unused actions from addMissingDefaults
 2022-02-15 11:01:29 +01:00
alesapin
447cd56cb9 Fix comments 2022-02-15 12:11:50 +03:00
alesapin
e15396d90c Fix race condition: 2022-02-14 22:19:49 +03:00
Nikolai Kochetov
b3ea360cd2 Fix a little bit more 2022-02-14 19:05:30 +00:00
Kseniia Sumarokova
382b8e0388
Merge pull request #34432 from ClickHouse/static-files-disk-uploader-create-symlinks 
`static-files-disk-uploader`: add a mode to create symlinks
 2022-02-14 18:10:53 +01:00
vdimir
99ca89c0ca
Fix StorageJoin and Asof or join_use_nulls in pipeline 2022-02-14 14:14:27 +00:00
alesapin
bb69455395
Merge pull request #34504 from CurtizJ/ttl-move-if-exists 
Support `TTL TO [DISK|VOLUME] [IF EXISTS]`
 2022-02-14 14:56:18 +03:00
alesapin
b75d551281 Fix clang tidy and add check for master 2022-02-14 14:37:41 +03:00
alesapin
b2886a429b Fix lock during fetch 2022-02-14 12:20:27 +03:00
alesapin
beb4400978 Fix 'same local part' check 2022-02-13 23:08:29 +03:00
alesapin
89373155fc Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-13 21:07:54 +03:00
Yakov Olkhovskiy
579fe6c97a major rework, transform added to the insert pipe 2022-02-13 17:42:59 +00:00
Maksim Kita
cb09aadc72
Merge pull request #34524 from FrankChen021/urlengine 
Fix compression support in URL engine
 2022-02-13 14:39:36 +01:00
Maksim Kita
ae1da31d19
Merge pull request #34558 from ucasfl/storage-log 
try fix data race in StorageLog
 2022-02-13 14:33:36 +01:00
mergify[bot]
8a47ed2b3f
Merge branch 'master' into static-files-disk-uploader-create-symlinks 2022-02-13 06:12:39 +00:00
alexey-milovidov
1774836d4f
Merge pull request #34555 from kitaisreal/table-functions-insert-partition-by-refactoring 
TableFunctionFile added performance test
 2022-02-13 09:08:51 +03:00
feng lv
4ccd1c3278 try fix data race in StorageLog 2022-02-13 04:52:16 +00:00
Maksim Kita
e2c8ba9ab2 Added performance test 2022-02-12 16:05:35 +00:00
alexey-milovidov
747b6b2058
Merge pull request #34528 from CurtizJ/key-condition-float 
Fix comparison between integers and floats in index analysis
 2022-02-12 10:19:09 +03:00
alexey-milovidov
ea71dc9d11
Merge pull request #34510 from kitaisreal/table-functions-insert-partition-by-refactoring 
Improve performance of insert into table functions URL, S3, File, HDFS
 2022-02-12 10:14:00 +03:00
alexey-milovidov
9bb2eba281
Merge pull request #34429 from ClickHouse/filimonov-SystemAsynchronousInserts 
asynchronous_inserts engine AsynchronousInserts -> SystemAsynchronousInserts
 2022-02-12 10:08:04 +03:00
Frank Chen
7b7bc8f6c3 Address review comment 2022-02-12 10:46:01 +08:00
Dmitry Novik
50131fd732 Fix cancelation for S3 and HDFS 2022-02-11 18:14:55 +00:00
Anton Popov
6a8e35930f fix comparison with integers and floats in index analysis 2022-02-11 18:20:37 +03:00
tavplubix
6d1fae834a
Merge pull request #34187 from qoega/default-table-engine 
Default table engine
 2022-02-11 18:14:29 +03:00
Anton Popov
2fcd69baf7 fix comparison with integers and floats in index analysis 2022-02-11 17:15:27 +03:00
mergify[bot]
f099048dd1
Merge branch 'master' into urlengine 2022-02-11 11:03:21 +00:00
Frank Chen
c8cddd7077 Fix compression for URL engine 2022-02-11 17:45:49 +08:00
Maksim Kita
13cbf79ecb Improve performance of insert into table functions URL, S3, File, HDFS 2022-02-10 20:06:23 +00:00
alesapin
705529ca03 Followup 2022-02-10 22:50:15 +03:00
alesapin
ef61c9b47c fix 2022-02-10 22:49:33 +03:00
alesapin
3af06b23f8 POC 2022-02-10 22:45:52 +03:00
alesapin
088f8cacf3 Small improvements 2022-02-10 19:50:21 +03:00
Anton Popov
70986a70a1 support TTL TO [DISK|VOLUME] [IF EXISTS] 2022-02-10 19:26:23 +03:00
Kruglov Pavel
bf8c028533
Merge pull request #34479 from Avogar/fix-filesystem-error 
Fix possible error 'file_size: Operation not supported'
 2022-02-10 14:17:05 +03:00
alesapin
f764da35ca Also zero copy mutations 2022-02-10 14:15:08 +03:00
Kruglov Pavel
a4f5610764
Merge pull request #34476 from CurtizJ/avoid-settings-copy 
Avoid unnecessary copying of `Settings`
 2022-02-10 14:13:46 +03:00
alesapin
70221b272b Better solution 2022-02-10 12:57:11 +03:00
Anton Popov
298838f891 avoid unnecessary copying of Settings 2022-02-10 12:13:51 +03:00
avogar
bfa96463ca Fix possible error 'file_size: Operation not supported' 2022-02-10 09:23:27 +03:00
alesapin
c587160308 Bad fix which will affect zk and should be changed 2022-02-09 23:06:44 +03:00
alesapin
57037465f5 Trying to fix tests blindly 2022-02-09 22:56:22 +03:00
Kruglov Pavel
538830931b
Merge branch 'master' into fix-url-globs 2022-02-09 20:09:56 +03:00
mergify[bot]
d55eadc3c9
Merge branch 'master' into default-table-engine 2022-02-09 17:08:31 +00:00
Kruglov Pavel
4ec8da73c4
Merge pull request #34448 from Avogar/fix-url-engine 
Fix bug in URL engine
 2022-02-09 19:19:05 +03:00
avogar
84087c1554 Improve schema inference with globs 2022-02-09 19:16:15 +03:00
Yakov Olkhovskiy
a134ab282b style fixed and minor optimization 2022-02-09 13:51:12 +00:00
Anton Popov
822b58247a
Merge pull request #34132 from CurtizJ/fix-distributed-inserts 
Fix inserts to distributed tables in case of change of native protocol
 2022-02-09 14:58:44 +03:00
Kruglov Pavel
e0e36c256e
Fix style 2022-02-09 14:44:46 +03:00
alesapin
10c3e6e546 Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-09 14:14:58 +03:00
alesapin
72863cc4c3 Fix error 2022-02-09 13:57:10 +03:00
avogar
7baf23f35f Fix bug URL engine 2022-02-09 13:53:54 +03:00
alesapin
437940b29d
Merge pull request #34422 from ClickHouse/add_settings_to_part_size_in_s3 
Add settings for multipart upload part size in s3
 2022-02-09 12:51:20 +03:00
Kseniia Sumarokova
86956ca08b
Merge pull request #34392 from kssenii/http-buffer-skip-not-found-url-for-globs 
Allow to skip not found urls for globs
 2022-02-09 09:32:09 +01:00
Yakov Olkhovskiy
b9777be271 insertion with ephemerals requires columns list, insertion without columns list omits ephemeral 2022-02-09 01:11:29 +00:00
Alexey Milovidov
742620209a static-files-disk-uploader: add mode to create symlinks 2022-02-09 03:50:50 +03:00
kssenii
e8a8f46864 Fix style check 2022-02-08 21:58:03 +01:00
alesapin
36909a986f Fix bug with files remove 2022-02-08 22:21:16 +03:00
filimonov
8d310e4ac3
AsynchronousInserts -> SystemAsynchronousInserts 2022-02-08 19:23:40 +01:00
alesapin
02a93cb852 Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-08 19:42:27 +03:00
alesapin
f6e7cea1f8 Add settings for S3 multipart upload parts size 2022-02-08 19:38:04 +03:00
kssenii
9f2628b8f2 Fix 2022-02-08 16:41:55 +01:00
Kruglov Pavel
cae1517693
Fix build 2022-02-08 17:31:02 +03:00
Nikolai Kochetov
0f7c0c72bd
Merge pull request #34305 from amosbird/projection-fix27 
Fix various issues when projection is enabled by default
 2022-02-08 17:19:56 +03:00
Kruglov Pavel
720310fcf1
Merge branch 'master' into http-buffer-skip-not-found-url-for-globs 2022-02-08 16:56:22 +03:00
Kruglov Pavel
3e216adc17
Merge pull request #34405 from Avogar/fix-segfault-in-url 
Fix segfault in schema inference from url
 2022-02-08 16:48:37 +03:00
Anton Popov
7e9770dcf0 minor enhancements 2022-02-08 15:57:23 +03:00
kssenii
dc5f035265 Fix 2022-02-08 13:28:10 +01:00
Nicolae Vartolomei
50ee264223 Disable projects when allow_experimental_query_deduplication is in use 2022-02-08 12:16:10 +00:00
alesapin
8e9ccbd077
Merge pull request #33933 from sunny19930321/fix-substr-zk-metadata 
Better local metadata comparison with ZooKeeper metadata
 2022-02-08 14:58:46 +03:00
alesapin
b47b0eb1dc Revert accident change 2022-02-08 14:05:01 +03:00
avogar
e118c89bb2 Fix segfault in schema inference from url 2022-02-08 13:40:03 +03:00
Yatsishin Ilya
bcf4303286 Merge remote-tracking branch 'origin' into default-table-engine 2022-02-08 10:35:29 +00:00
kssenii
3834bdbae0 Fixes 2022-02-08 11:11:18 +01:00
alesapin
3af6012cb4 Revert "Revert "Revert "Revert "Merge pull request #34219 from ClickHouse/revert-34212-revert-33291-add-pool-to-s3-write-buffer"""" 
This reverts commit 2bc2ea485e.
 2022-02-08 11:01:26 +03:00
alesapin
2bc2ea485e Revert "Revert "Revert "Merge pull request #34219 from ClickHouse/revert-34212-revert-33291-add-pool-to-s3-write-buffer""" 
This reverts commit fb77d7a7d5.
 2022-02-08 10:56:29 +03:00
alexey-milovidov
43ee8ddb5b
Merge pull request #34376 from CurtizJ/fix-reading-empty-arrays 
Fix consecutive backward seeks in seekable read buffers
 2022-02-08 02:27:57 +03:00
Yakov Olkhovskiy
4d5fb56c29 added EPHEMERAL default for column (preliminary) 2022-02-07 23:21:10 +00:00
kssenii
eba3011ada Fix 2022-02-07 20:40:47 +01:00
Nicolae Vartolomei
7d77678a9f Remove useless setting experimental_query_deduplication_send_all_part_uuids 
This setting made sense for testing deduplication before part movement was actually implemented.

allow_experimental_query_deduplication setting is enough and code is covered by test_part_moves_between_shards
 2022-02-07 19:03:20 +00:00
Anton Popov
1b16db72c3 fix consecutive backward seeks in seekable read buffers 2022-02-07 17:20:26 +03:00
alesapin
523d1059ed
Update KeyDescription.cpp 2022-02-07 13:59:03 +03:00
alesapin
ba28c2c013 Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-07 12:44:56 +03:00
alesapin
fb77d7a7d5 Revert "Revert "Merge pull request #34219 from ClickHouse/revert-34212-revert-33291-add-pool-to-s3-write-buffer"" 
This reverts commit 875e5413ad.
 2022-02-07 12:36:54 +03:00
alesapin
bb361e3b01 Fix bug with existing part 2022-02-07 12:33:37 +03:00
Yatsishin Ilya
e0803064e6 Merge remote-tracking branch 'origin' into default-table-engine 2022-02-07 08:45:32 +00:00
Amos Bird
2debfc922d
Better projection format and test fixes 2022-02-07 10:47:11 +08:00
alexey-milovidov
fb9ed9acad
Merge pull request #34175 from kitaisreal/bitset-sort-performance-check 
bitsetsort peformance check
 2022-02-07 05:34:57 +03:00
feng lv
6325d4d9b0 continue of #34317 
fix

fix
 2022-02-06 08:59:17 +00:00
Amos Bird
52aabf98fe
Revise and add more comments 2022-02-06 16:53:54 +08:00
Amos Bird
1ab773cc90
Fix aggregation_in_order with normal projection 2022-02-06 16:46:12 +08:00
Amos Bird
3fab7af541
Bug fix and improvement of minmax_count_projection 2022-02-06 16:46:11 +08:00
Amos Bird
a0ab7a01f1
Adapt minmax_count_projection with ModuleLegacy 2022-02-06 16:46:11 +08:00
Amos Bird
98857de82b
Disable projection for high-order storages 2022-02-06 16:46:10 +08:00
Amos Bird
27fcefd315
Disable projection when doing parallel replica reading 2022-02-06 16:46:10 +08:00
Amos Bird
7674bc986e
Disable projection when there is JOIN or SAMPLE 2022-02-06 16:46:09 +08:00
Maksim Kita
e626b7b2d3 Fixed tests 2022-02-04 19:52:18 +00:00
Maksim Kita
1cd1fb9658 Fixed tests 2022-02-04 19:52:18 +00:00
Maksim Kita
f301e50b23 Fixed tests 2022-02-04 19:52:18 +00:00
alexey-milovidov
e194bf3377
Merge pull request #34317 from ucasfl/lowcard 
use LowCardinality for _file and _path virtual columns in StorageFile
 2022-02-04 21:39:44 +03:00
Nikolai Kochetov
daeeb6f3a2
Merge pull request #34316 from ClickHouse/probably-fix-data-race-in-WriteBufferFromS3 
Probably fix data race in WriteBufferFromS3 destructor.
 2022-02-04 21:04:46 +03:00
Nikolai Kochetov
6436024e08 Fix test with ttl. 2022-02-04 16:05:02 +00:00
alesapin
c028269e6f Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-04 18:02:25 +03:00
alesapin
d4d147abab Remove redundant 2022-02-04 18:01:49 +03:00
Azat Khuzhin
63e674280b Decrease severity for "Reading ... ranges ..." log message to Trace 
That way with send_logs_level='debug', you will not get verbose
information that you already has, since there is summary row:

    Selected ... parts by partition key, ... parts by primary key, ... marks by primary key, ... marks to read from ... ranges

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-04 18:00:59 +03:00
tavplubix
22de534fdc
Merge pull request #34297 from ClickHouse/fix_restarting_thread 
Try to fix race between pullLogsToQueue and RestartingThread
 2022-02-04 17:30:17 +03:00
feng lv
7bf2243431 use LowCardinality for _file and _path virtual columns in StorageFile 
fix
 2022-02-04 14:15:56 +00:00
alesapin
1582c4bf24 Fix mutaion status for retryable errors 2022-02-04 16:27:59 +03:00
Maksim Kita
ab696e6b59
Merge pull request #34310 from ClickHouse/fix-parallel-loading-parts 
Fix parallel loading of data parts
 2022-02-04 13:10:24 +01:00
alesapin
875e5413ad Revert "Merge pull request #34219 from ClickHouse/revert-34212-revert-33291-add-pool-to-s3-write-buffer" 
This reverts commit b92efed350, reversing
changes made to ecce006cb2.
 2022-02-04 14:30:33 +03:00
alesapin
af18905a33 Fixup 2022-02-04 13:03:13 +03:00
alesapin
2ed45b2a98 Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-04 11:23:46 +03:00
Nikolai Kochetov
b92efed350
Merge pull request #34219 from ClickHouse/revert-34212-revert-33291-add-pool-to-s3-write-buffer 
Revert "Revert "Add pool to WriteBufferFromS3""
 2022-02-04 11:00:29 +03:00
Alexey Milovidov
78eebd5c7c Fix parallel loading of data parts 2022-02-04 02:29:46 +03:00
Alexander Tokmakov
897e94c16c make restarting thread less bad 2022-02-03 23:29:24 +03:00
Anton Popov
9b844c6b42
Merge pull request #32748 from CurtizJ/read-in-order-fixed-prefix 
Support `optimize_read_in_order` if prefix of sorting key is already sorted
 2022-02-03 18:17:08 +03:00
alesapin
2a9bc7cba8 Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-03 16:20:06 +03:00
Nikolai Kochetov
a2a9c10617
Merge pull request #34221 from ClickHouse/revert-34201-revert-34189-less-logging-for-remote_fs_execute_merges_on_single_replica_time_threshold 
Revert "Revert "Additionally check remote_fs_execute_merges_on_single_replica_time_threshold inside ReplicatedMergeTreeQueue""
 2022-02-03 13:09:44 +03:00
alesapin
763dfd7895 Fix write ahead log 2022-02-03 12:14:00 +03:00
alesapin
25375bc76b Remove unused param 2022-02-03 11:21:19 +03:00
alesapin
e0f640dd9f Fix 2022-02-02 19:44:29 +03:00
alesapin
80800e051e Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-02 19:42:04 +03:00
Nikolai Kochetov
ea044fc6b2 Merge branch 'master' into revert-34212-revert-33291-add-pool-to-s3-write-buffer 2022-02-02 19:40:41 +03:00
alesapin
b9c118524f Fix race condition on hardlink/erase/read metadata 2022-02-02 19:40:21 +03:00
Nikolai Kochetov
1117e6095d
Update src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp 
Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com>
 2022-02-02 18:34:05 +03:00
Azat Khuzhin
15993cb13b Add missing fmt::runtime() in MergeTreeBackgroundExecutor 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-02 11:37:53 +03:00
alesapin
47d538b52f
Merge pull request #34232 from ClickHouse/cancelled-merging-parts-error-message 
Change severity of the "Cancelled merging parts" message in logs
 2022-02-02 10:29:07 +03:00
Sergei Trifonov
68bc456830
Merge pull request #34223 from azat/bump-fmt 
Bump fmtlib from 7.0.0 to 8.1.1
 2022-02-02 00:03:25 +03:00
Yatsishin Ilya
759574d6f0 improve 2022-02-01 16:59:41 +00:00
Nikolai Kochetov
dd81eff301 Fix tests. 2022-02-01 15:28:22 +00:00