Commit Graph

7881 Commits

Author SHA1 Message Date
Vitaly Baranov
641a5f5e35
Merge pull request #36198 from vitlibar/backup-improvements-3
Backup for replicated tables and other improvements
2022-04-28 09:23:53 +02:00
mergify[bot]
0d541d2da6
Merge branch 'master' into less-stat-calls 2022-04-28 01:50:49 +00:00
Anton Popov
eb810116a4 fix vertical merges in wide parts 2022-04-27 13:42:47 +00:00
taiyang-li
0341880250 Merge remote-tracking branch 'origin/master' into hive_text_parallel_parsing 2022-04-27 11:15:16 +08:00
Alexey Milovidov
b8a9f1411a
Merge pull request #36649 from azat/system.processes-is_all_data_sent
clickhouse-test: improve left queries after the test hardening
2022-04-27 05:21:22 +03:00
HeenaBansal2009
4961caec13 Merge branch 'master' into Fix-Coverity-Issues 2022-04-26 11:43:44 -07:00
Alexander Tokmakov
b1949ee901
Merge pull request #36680 from ClickHouse/better_logs_virtual_parts
Better logs for virtual parts
2022-04-26 21:36:20 +03:00
Alexander Tokmakov
d149ac8bd7 better logs for virtual parts 2022-04-26 16:58:40 +02:00
HeenaBansal2009
13a9a8fd5e Style check fix 2022-04-26 07:13:09 -07:00
HeenaBansal2009
267f426605 Minor Coverity defetcs fixes 2022-04-26 07:13:09 -07:00
kssenii
9d364cdce2 Refactor 2022-04-26 15:33:53 +02:00
Alexander Tokmakov
190becdcc3
Merge pull request #36644 from ClickHouse/fix_some_flaky_tests
Fix some flaky tests
2022-04-26 13:40:14 +03:00
Sergei Trifonov
c1a0165f7d merge from master and fix conflicts 2022-04-26 12:39:16 +02:00
Azat Khuzhin
6339a48923 Add is_all_data_sent into system.processes
v2: fix SHOW PROCESSLIST (does not have process list entry)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-26 12:15:00 +03:00
Anton Popov
e051751012
Merge pull request #36637 from CurtizJ/dynamic-columns-11
Fix merges of wide parts with type `Object`
2022-04-26 10:34:04 +02:00
taiyang-li
b7cc344d62 remove useless codes 2022-04-26 14:42:43 +08:00
taiyang-li
99dee35b6e parallel parsing of hive text format 2022-04-26 14:33:10 +08:00
Alexey Milovidov
7683901689
Merge pull request #36630 from azat/mutator-cleanup
Tiny Mutator code cleanup
2022-04-26 08:24:50 +03:00
Alexey Milovidov
02662dfd13
Update ColumnsDescription.h 2022-04-26 08:23:01 +03:00
Memo
c38a4b4255
Update src/Storages/MergeTree/MergeTreeData.h
Co-authored-by: alesapin <alesapin@gmail.com>
2022-04-26 10:26:18 +08:00
Anton Popov
866a51a948 reduce number of stat calls in storage File 2022-04-25 22:13:15 +00:00
mergify[bot]
a5aab53b70
Merge branch 'master' into add_query_level_settingss 2022-04-25 21:41:36 +00:00
mergify[bot]
178f244c8b
Merge branch 'master' into dynamic-columns-11 2022-04-25 21:36:56 +00:00
alesapin
92296484e7
Merge pull request #36348 from rschu1ze/erase_if3
Replace remove-erase idiom by C++20 erase()/erase_if()
2022-04-25 23:34:18 +02:00
Alexander Tokmakov
19a376b9f7 fix assertions 2022-04-25 22:41:46 +02:00
Anton Popov
d28b1559e6 fix merges of wide parts with type Object 2022-04-25 16:44:24 +00:00
mergify[bot]
265398d1b6
Merge branch 'master' into feat/add_part_offset 2022-04-25 15:58:16 +00:00
Azat Khuzhin
8b544e26d3 Move some functions from MergeTreeDataMergerMutator to MutateTask
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-25 18:16:38 +03:00
Azat Khuzhin
f4df88a938 Add a comment for ColumnsDescription::subcolumns
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-25 18:16:38 +03:00
Azat Khuzhin
615d35e233 Remove dead function MergeTreeDataMergerMutator::splitMutationCommands()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-25 18:16:38 +03:00
Azat Khuzhin
8440fec3db Fix coding alignment in MutateTask
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-25 18:16:38 +03:00
Vitaly Baranov
68a020ecea Implement BACKUP/RESTORE ON CLUSTER. 2022-04-25 16:34:33 +02:00
Vitaly Baranov
144d3aefeb Add system table system.backups; implement async mode for commands BACKUP & RESTORE 2022-04-25 16:34:33 +02:00
Vitaly Baranov
3966ee1e30 Use SeekableReadBuffer instead of ReadBuffer in IBackupEntry. 2022-04-25 16:34:33 +02:00
Vitaly Baranov
acd28d8a1d Implement RESTORE for replicated tables. 2022-04-25 16:34:33 +02:00
Nikolai Kochetov
0fa63a8d65
Merge pull request #36549 from ClickHouse/interpolate-alias-fix
Avoid projections if INTERPOLATE with expressions is present
2022-04-25 13:18:00 +02:00
Kruglov Pavel
34c342fdd3
Merge pull request #36205 from Avogar/improve-globs
Some refactoring around schema inference with globs
2022-04-25 13:14:46 +02:00
Nikolai Kochetov
d5b2d28f7b
Merge pull request #36579 from amosbird/refactor-executor1
Small refactoring of Processors and QueryPipeline
2022-04-25 12:47:59 +02:00
Alexander Gololobov
c484a1c269 Preserve order of virtual columns in the header and in data chunks when adding _part_offset together with other virtual columns 2022-04-25 12:10:56 +02:00
alesapin
edc78247f0
Merge pull request #36299 from ClickHouse/fix_trash
Trying to fix some trash in zero copy replication
2022-04-24 13:19:43 +02:00
Amos Bird
e73d7fea31
Rename narrowBlockInputStream to narrowPipe 2022-04-24 18:33:48 +08:00
tavplubix
9fa1edd6ef
Merge pull request #36535 from xiedeyantu/fix-log
fix log should print 'from' path
2022-04-23 14:10:09 +03:00
alesapin
46792379e7 Merge branch 'master' into fix_trash 2022-04-23 12:05:35 +02:00
alesapin
2b04c19435 Satisfy clang-tidy 2022-04-23 11:58:56 +02:00
alesapin
f183eabaa2 Return normal settings 2022-04-22 22:36:55 +02:00
alesapin
ebdd53ef8f Fix shared fetch 2022-04-22 19:18:18 +02:00
Nikita Mikhaylov
e475849761 Style and Docs update 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
bf6d3af536 Better 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
31ccb9c1c3 Done 2022-04-22 13:56:16 +00:00
Yakov Olkhovskiy
e4af79c2a0 avoid projections if INTERPOLATE with expressions is present 2022-04-22 09:47:59 -04:00
alesapin
2559d62d74 Better logging and bugfixes 2022-04-22 15:35:25 +02:00
alesapin
64383bcb0a Better 2022-04-22 15:35:25 +02:00
Antonio Andelic
2e187e4b97
Merge pull request #36439 from xiedeyantu/temp-table-improve
temporary table can show total rows and total bytes
2022-04-22 13:41:25 +02:00
Kseniia Sumarokova
33bb48106f
Merge pull request #36314 from CurtizJ/print-bad-filenames
Show names of erroneous files in case of parsing errors while executing table functions
2022-04-22 13:24:55 +02:00
xiedeyantu
550460c541 fix log should print from path 2022-04-22 18:30:40 +08:00
Sergei Trifonov
b371251e80 Merge branch 'master' into surahman-CH-31221-AWS-S3-object-version-id 2022-04-22 08:39:02 +02:00
mergify[bot]
cd8fa75546
Merge branch 'master' into fix_trash 2022-04-21 21:45:31 +00:00
Alexey Milovidov
b1e3130a5e
Revert "Minor refactor to prefer C++ Standard Algorithms" 2022-04-21 20:22:37 +03:00
Mikhail f. Shiryaev
e3d8da26b7
Merge pull request #36502 from ClickHouse/22.5-prepare
Update version after release
2022-04-21 17:44:09 +02:00
Mikhail f. Shiryaev
5997d7af60
Update version to 22.5.1.1 2022-04-21 17:39:40 +02:00
Maksim Kita
57444fc7d3
Merge pull request #36444 from rschu1ze/clang-tidy-fixes
Clang tidy fixes
2022-04-21 16:11:27 +02:00
Dmitry Novik
77a82cc090
Merge pull request #35631 from amosbird/projection-fix1
Fix broken SET reuse during projection analysis.
2022-04-21 15:32:52 +02:00
mergify[bot]
04e3356d50
Merge branch 'master' into temp-table-improve 2022-04-21 12:43:55 +00:00
alesapin
5465415751 Fix replace/move partition with zero copy replication 2022-04-21 14:39:12 +02:00
mergify[bot]
1ba1cad5cf
Merge branch 'master' into improve-globs 2022-04-21 11:52:13 +00:00
Kruglov Pavel
7fb0f2284d
Merge branch 'master' into fix-s3-cluster 2022-04-21 13:39:52 +02:00
alesapin
cbfe00bb47 Fix tidy 2022-04-21 00:30:13 +02:00
alesapin
c14e2e0b96 Fix more 2022-04-20 21:08:26 +02:00
Sergei Trifonov
34842f209c merge master and fix conflicts 2022-04-20 18:35:56 +02:00
Kseniia Sumarokova
842879beea
Merge pull request #36272 from kssenii/mongo-fix
Allow missing columns for mongo storage
2022-04-20 17:54:35 +02:00
xiedeyantu
8d4a104d22 fix code 2022-04-20 23:43:46 +08:00
Kruglov Pavel
56cfa55bd5
Fix style 2022-04-20 16:35:57 +02:00
Kruglov Pavel
813e228fcc
Merge branch 'master' into improve-globs 2022-04-20 16:31:47 +02:00
alesapin
6ee0df251f Add ditry settings 2022-04-20 15:11:03 +02:00
alesapin
829854113b Add a little thread safety 2022-04-20 15:10:36 +02:00
alesapin
e8fba1644c Fix bug 2022-04-20 14:13:29 +02:00
alesapin
41b57f78cf Revert extreme settings 2022-04-20 13:59:12 +02:00
alesapin
5334522d5c Extreme settings 2022-04-20 13:33:42 +02:00
alesapin
2eee79dc2a Review fixes 2022-04-20 13:26:20 +02:00
alesapin
40c15222f8 Merge branch 'master' into fix_trash 2022-04-20 12:45:49 +02:00
Antonio Andelic
d77678d1d1
Merge pull request #36410 from xinhuitian/move_down_data_range_construction
Move down data_range construction in filterMarksUsingIndex and filterMarksUsingMergedIndex
2022-04-20 12:18:05 +02:00
tavplubix
2c54381520
Merge pull request #36200 from ClickHouse/fix_create_as_replicated_default_args
Fix CREATE AS for ReplicatedMergeTree with default arguments
2022-04-20 12:01:13 +03:00
Robert Schulze
b24ca8de52
Fix various clang-tidy warnings
When I tried to add cool new clang-tidy 14 warnings, I noticed that the
current clang-tidy settings already produce a ton of warnings. This
commit addresses many of these. Almost all of them were non-critical,
i.e. C vs. C++ style casts.
2022-04-20 10:29:05 +02:00
Kseniia Sumarokova
766a84e64f
Merge pull request #35915 from kssenii/better-s3-settings
Better settings configuration for s3
2022-04-20 10:22:06 +02:00
Robert Schulze
b6d7367538
Merge remote-tracking branch 'origin/master' into erase_if3
Conflicts:
- Interpreters/ActionsDAG.cpp
2022-04-20 10:02:59 +02:00
Antonio Andelic
f0774ca9ee
Merge pull request #36393 from tcoyvwac/fix/prefer-standard-algorithms-std-transform-storages
Minor refactor to prefer C++ Standard Algorithms
2022-04-20 08:42:54 +02:00
Antonio Andelic
bbb0be6a44
Merge pull request #36402 from rschu1ze/clang-tidy-contains
Activate clang-tidy warning "readability-container-contains"
2022-04-20 08:15:46 +02:00
xiedeyantu
47c826935d temporary table can show total rows and total bytes 2022-04-20 10:03:16 +08:00
avogar
f31f019252 Fix 2022-04-19 19:25:41 +00:00
avogar
1f252cedfe Make better 2022-04-19 19:16:47 +00:00
avogar
9a81b67ba5 Fix bug in s3Cluster schema inference 2022-04-19 18:23:04 +00:00
Anton Popov
bee4ca9b62 add more tests for error diagnostics in files 2022-04-19 15:56:34 +00:00
alesapin
aea7c7755a Fix style 2022-04-19 17:25:08 +02:00
Anton Popov
3e361c9759 Merge remote-tracking branch 'upstream/master' into HEAD 2022-04-19 14:18:04 +00:00
alesapin
7919485db2 Fix 2022-04-19 15:57:25 +02:00
alesapin
7cb7c120cc Less ugly 2022-04-19 15:53:10 +02:00
Anton Popov
056500f2ad
Merge pull request #33391 from zhongyuankai/fix_storage_distributed_ttl
Fix Alter ttl modification unsupported table engine
2022-04-19 15:47:01 +02:00
alesapin
cc06bc3d99 Add some clarifications 2022-04-19 14:01:30 +02:00
tcoyvwac
2441d33940
storages: prefer standard algorithms
StorageSystemZooKeeper.cpp:
Prefer std::any_of over ranged-for.
2022-04-19 13:40:50 +02:00
tcoyvwac
f7789c9b6d
storages: prefer standard algorithms
StorageSystemZooKeeper.cpp:
Prefer std::transform over ranged-for.
2022-04-19 13:40:49 +02:00
tcoyvwac
10dfeedd23
storages: prefer standard algorithms
StorageSystemZooKeeper.cpp:
Prefer std::transform over ranged-for.
2022-04-19 13:40:49 +02:00
alesapin
ee8d26ff0e Better test and fix for move 2022-04-19 13:27:55 +02:00
Sergei Trifonov
75c1053ad7 Merge remote-tracking branch 'origin/master' into surahman-CH-31221-AWS-S3-object-version-id 2022-04-19 13:17:25 +02:00
alesapin
8ade9fba52 Fix 2022-04-19 11:56:14 +02:00
mergify[bot]
d894aed7ae
Merge branch 'master' into mongo-fix 2022-04-19 09:32:51 +00:00
kssenii
0e05e1fa2f Fix 2022-04-19 11:31:27 +02:00
kssenii
1f9a597b77 Merge master 2022-04-19 11:31:13 +02:00
kssenii
f8f66dd23d Better 2022-04-19 10:59:47 +02:00
xinhuitian
5d0957e669 move down data range construction in filterMarksUsingIndex and filterMarksUsingMergedIndex 2022-04-19 13:26:33 +08:00
alesapin
c81cb9e563 Better 2022-04-19 01:12:07 +02:00
alesapin
bd7b3847c1 Some code 2022-04-19 01:09:09 +02:00
Robert Schulze
118e94523c
Activate clang-tidy warning "readability-container-contains"
This check suggests replacing <Container>.count() by
<Container>.contains() which is more speaking and in case of
multimaps/multisets also faster.
2022-04-18 23:53:11 +02:00
mergify[bot]
daca157111
Merge branch 'master' into fix_storage_distributed_ttl 2022-04-18 17:31:59 +00:00
Memo
a17f102fe6 fix style 2022-04-18 20:31:40 +08:00
alesapin
a97754f462 Fix 2022-04-18 13:39:09 +02:00
alesapin
c4b70f3ed3 Merge branch 'master' into fix_trash 2022-04-18 12:20:09 +02:00
zhanglistar
5670dcf750 Fix crash in debug mode of ReadBufferFromHDFS. 2022-04-18 17:52:59 +08:00
zhanglistar
355333b361
Merge branch 'ClickHouse:master' into fixhdfsreadbuffer 2022-04-18 16:42:03 +08:00
Memo
b3adf150b5 add_query_level_settings 2022-04-18 12:15:41 +08:00
Alexey Milovidov
f6ab2bd523
Merge pull request #36312 from ClickHouse/remove-arcadia
Remove remaining parts of Arcadia
2022-04-18 07:02:54 +03:00
Alexey Milovidov
36595e4206
Merge pull request #36320 from ClickHouse/fix-clang-tidy-14
Fix clang-tidy-14 (part 1)
2022-04-18 07:02:10 +03:00
Alexey Milovidov
242919eddd Remove abbreviation 2022-04-18 01:02:49 +02:00
Robert Schulze
1e1df8e101
Replace remove-erase idiom by C++20 erase()/erase_if()
- makes the code less verbose while being as efficient
2022-04-17 12:04:47 +02:00
Alexey Milovidov
294efeccfe Fix clang-tidy-14 (part 1) 2022-04-16 04:54:04 +02:00
Anton Popov
2de6668b3f show names of erroneous files 2022-04-16 00:10:47 +00:00
Alexey Milovidov
cbeeb7ec4f Remove Arcadia 2022-04-16 00:20:47 +02:00
alesapin
1706ae9e15 Some trash implementation 2022-04-15 18:36:23 +02:00
alesapin
5a8419a48e Remove more trash 2022-04-15 17:05:17 +02:00
alesapin
eb7593f786 Remove more trash 2022-04-15 16:24:38 +02:00
zhanglistar
fe93fd83f5
Merge branch 'ClickHouse:master' into fixhdfsreadbuffer 2022-04-15 16:27:20 +08:00
zhanglistar
e22a175afb Fix ReadBufferFromHDFS crash in debug mode 2022-04-15 14:04:30 +08:00
Anton Popov
305dd57262
Merge branch 'master' into fix_storage_distributed_ttl 2022-04-14 14:51:15 +02:00
tavplubix
3af2384066
Merge pull request #36190 from azat/revert-27002-mutation-stuck
Revert "Fix possible mutation stuck due to race with DROP_RANGE"
2022-04-14 15:03:35 +03:00
alesapin
bd629f4cd0
Merge pull request #36163 from ClickHouse/maybe_fix_36098
Some fixes for ReplicatedMergeTree
2022-04-14 12:12:55 +02:00
tavplubix
fd23c42ea7
Update StorageReplicatedMergeTree.cpp 2022-04-13 22:33:55 +03:00
avogar
42726639f3 Check ORC/Parquet/Arrow format magic bytes before loading file in memory 2022-04-13 19:27:38 +00:00
avogar
1c065f8c7a Some refactoring around schema inference with globs 2022-04-13 17:02:48 +00:00
Alexander Tokmakov
66fdf35dfd remove outdated parts immediately on drop partition 2022-04-13 18:01:22 +02:00
Alexander Gololobov
fcb83a12ff
Merge pull request #35953 from azat/ttl-mutations
Require mutations for per-table TTL only when it had been changed
2022-04-13 17:14:04 +02:00
Alexander Tokmakov
40fea2b833 do not unfold {uuid} macro 2022-04-13 16:51:59 +02:00
kssenii
0912b038fa Merge master 2022-04-13 15:30:01 +02:00
Azat Khuzhin
6dbc138e76 Remove duplicated parts removal from mutation.
This commit reverts 701e2ffd79 ("Fix
possible mutation stuck due to race with DROP_RANGE"), and also adds
some clarification comments.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-13 16:09:17 +03:00
Sergei Trifonov
9d1a2da685 merge master and fix conflicts 2022-04-13 14:53:36 +02:00
Julian Gilyadov
a4f56f3330
Throw exception when file cant be executed instead of displaying success 2022-04-12 17:52:44 +02:00
mergify[bot]
0b3c15c07a
Merge branch 'master' into projection-fix1 2022-04-12 13:49:28 +00:00
Alexander Tokmakov
457a9e9691 fixes for ReplicatedMergeTree 2022-04-12 14:14:26 +02:00
Vladimir C
8873d7b0e7
Merge pull request #36082 from bigo-sg/cache_hive_files 2022-04-12 11:42:08 +02:00
Kseniia Sumarokova
b29202ac88
Merge pull request #36150 from kssenii/fix-cache-test
Fix test 02241_remote_filesystem_cache_on_insert for database Ordinary
2022-04-12 11:08:44 +02:00
taiyang-li
9d31c44d39 fix bug of read buffer from hdfs 2022-04-12 12:01:51 +08:00
Kseniia Sumarokova
51dd39b42f
Update StorageSystemRemoteDataPaths.cpp 2022-04-11 22:46:51 +02:00