Commit Graph

8085 Commits

Author SHA1 Message Date
Anton Popov
e051751012
Merge pull request #36637 from CurtizJ/dynamic-columns-11
Fix merges of wide parts with type `Object`
2022-04-26 10:34:04 +02:00
taiyang-li
b7cc344d62 remove useless codes 2022-04-26 14:42:43 +08:00
taiyang-li
99dee35b6e parallel parsing of hive text format 2022-04-26 14:33:10 +08:00
Alexey Milovidov
7683901689
Merge pull request #36630 from azat/mutator-cleanup
Tiny Mutator code cleanup
2022-04-26 08:24:50 +03:00
Alexey Milovidov
02662dfd13
Update ColumnsDescription.h 2022-04-26 08:23:01 +03:00
Memo
c38a4b4255
Update src/Storages/MergeTree/MergeTreeData.h
Co-authored-by: alesapin <alesapin@gmail.com>
2022-04-26 10:26:18 +08:00
Anton Popov
866a51a948 reduce number of stat calls in storage File 2022-04-25 22:13:15 +00:00
mergify[bot]
a5aab53b70
Merge branch 'master' into add_query_level_settingss 2022-04-25 21:41:36 +00:00
mergify[bot]
178f244c8b
Merge branch 'master' into dynamic-columns-11 2022-04-25 21:36:56 +00:00
alesapin
92296484e7
Merge pull request #36348 from rschu1ze/erase_if3
Replace remove-erase idiom by C++20 erase()/erase_if()
2022-04-25 23:34:18 +02:00
Alexander Tokmakov
19a376b9f7 fix assertions 2022-04-25 22:41:46 +02:00
Anton Popov
d28b1559e6 fix merges of wide parts with type Object 2022-04-25 16:44:24 +00:00
mergify[bot]
265398d1b6
Merge branch 'master' into feat/add_part_offset 2022-04-25 15:58:16 +00:00
Azat Khuzhin
8b544e26d3 Move some functions from MergeTreeDataMergerMutator to MutateTask
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-25 18:16:38 +03:00
Azat Khuzhin
f4df88a938 Add a comment for ColumnsDescription::subcolumns
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-25 18:16:38 +03:00
Azat Khuzhin
615d35e233 Remove dead function MergeTreeDataMergerMutator::splitMutationCommands()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-25 18:16:38 +03:00
Azat Khuzhin
8440fec3db Fix coding alignment in MutateTask
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-25 18:16:38 +03:00
Vitaly Baranov
68a020ecea Implement BACKUP/RESTORE ON CLUSTER. 2022-04-25 16:34:33 +02:00
Vitaly Baranov
144d3aefeb Add system table system.backups; implement async mode for commands BACKUP & RESTORE 2022-04-25 16:34:33 +02:00
Vitaly Baranov
3966ee1e30 Use SeekableReadBuffer instead of ReadBuffer in IBackupEntry. 2022-04-25 16:34:33 +02:00
Vitaly Baranov
acd28d8a1d Implement RESTORE for replicated tables. 2022-04-25 16:34:33 +02:00
Nikolai Kochetov
0fa63a8d65
Merge pull request #36549 from ClickHouse/interpolate-alias-fix
Avoid projections if INTERPOLATE with expressions is present
2022-04-25 13:18:00 +02:00
Kruglov Pavel
34c342fdd3
Merge pull request #36205 from Avogar/improve-globs
Some refactoring around schema inference with globs
2022-04-25 13:14:46 +02:00
Nikolai Kochetov
d5b2d28f7b
Merge pull request #36579 from amosbird/refactor-executor1
Small refactoring of Processors and QueryPipeline
2022-04-25 12:47:59 +02:00
Alexander Gololobov
c484a1c269 Preserve order of virtual columns in the header and in data chunks when adding _part_offset together with other virtual columns 2022-04-25 12:10:56 +02:00
alesapin
edc78247f0
Merge pull request #36299 from ClickHouse/fix_trash
Trying to fix some trash in zero copy replication
2022-04-24 13:19:43 +02:00
Amos Bird
e73d7fea31
Rename narrowBlockInputStream to narrowPipe 2022-04-24 18:33:48 +08:00
tavplubix
9fa1edd6ef
Merge pull request #36535 from xiedeyantu/fix-log
fix log should print 'from' path
2022-04-23 14:10:09 +03:00
alesapin
46792379e7 Merge branch 'master' into fix_trash 2022-04-23 12:05:35 +02:00
alesapin
2b04c19435 Satisfy clang-tidy 2022-04-23 11:58:56 +02:00
alesapin
f183eabaa2 Return normal settings 2022-04-22 22:36:55 +02:00
alesapin
ebdd53ef8f Fix shared fetch 2022-04-22 19:18:18 +02:00
Nikita Mikhaylov
e475849761 Style and Docs update 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
bf6d3af536 Better 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
31ccb9c1c3 Done 2022-04-22 13:56:16 +00:00
Yakov Olkhovskiy
e4af79c2a0 avoid projections if INTERPOLATE with expressions is present 2022-04-22 09:47:59 -04:00
alesapin
2559d62d74 Better logging and bugfixes 2022-04-22 15:35:25 +02:00
alesapin
64383bcb0a Better 2022-04-22 15:35:25 +02:00
Antonio Andelic
2e187e4b97
Merge pull request #36439 from xiedeyantu/temp-table-improve
temporary table can show total rows and total bytes
2022-04-22 13:41:25 +02:00
Kseniia Sumarokova
33bb48106f
Merge pull request #36314 from CurtizJ/print-bad-filenames
Show names of erroneous files in case of parsing errors while executing table functions
2022-04-22 13:24:55 +02:00
xiedeyantu
550460c541 fix log should print from path 2022-04-22 18:30:40 +08:00
Sergei Trifonov
b371251e80 Merge branch 'master' into surahman-CH-31221-AWS-S3-object-version-id 2022-04-22 08:39:02 +02:00
mergify[bot]
cd8fa75546
Merge branch 'master' into fix_trash 2022-04-21 21:45:31 +00:00
Alexey Milovidov
b1e3130a5e
Revert "Minor refactor to prefer C++ Standard Algorithms" 2022-04-21 20:22:37 +03:00
Mikhail f. Shiryaev
e3d8da26b7
Merge pull request #36502 from ClickHouse/22.5-prepare
Update version after release
2022-04-21 17:44:09 +02:00
Mikhail f. Shiryaev
5997d7af60
Update version to 22.5.1.1 2022-04-21 17:39:40 +02:00
Maksim Kita
57444fc7d3
Merge pull request #36444 from rschu1ze/clang-tidy-fixes
Clang tidy fixes
2022-04-21 16:11:27 +02:00
Dmitry Novik
77a82cc090
Merge pull request #35631 from amosbird/projection-fix1
Fix broken SET reuse during projection analysis.
2022-04-21 15:32:52 +02:00
mergify[bot]
04e3356d50
Merge branch 'master' into temp-table-improve 2022-04-21 12:43:55 +00:00
alesapin
5465415751 Fix replace/move partition with zero copy replication 2022-04-21 14:39:12 +02:00
mergify[bot]
1ba1cad5cf
Merge branch 'master' into improve-globs 2022-04-21 11:52:13 +00:00
Kruglov Pavel
7fb0f2284d
Merge branch 'master' into fix-s3-cluster 2022-04-21 13:39:52 +02:00
alesapin
cbfe00bb47 Fix tidy 2022-04-21 00:30:13 +02:00
alesapin
c14e2e0b96 Fix more 2022-04-20 21:08:26 +02:00
Sergei Trifonov
34842f209c merge master and fix conflicts 2022-04-20 18:35:56 +02:00
Kseniia Sumarokova
842879beea
Merge pull request #36272 from kssenii/mongo-fix
Allow missing columns for mongo storage
2022-04-20 17:54:35 +02:00
xiedeyantu
8d4a104d22 fix code 2022-04-20 23:43:46 +08:00
Kruglov Pavel
56cfa55bd5
Fix style 2022-04-20 16:35:57 +02:00
Kruglov Pavel
813e228fcc
Merge branch 'master' into improve-globs 2022-04-20 16:31:47 +02:00
alesapin
6ee0df251f Add ditry settings 2022-04-20 15:11:03 +02:00
alesapin
829854113b Add a little thread safety 2022-04-20 15:10:36 +02:00
alesapin
e8fba1644c Fix bug 2022-04-20 14:13:29 +02:00
alesapin
41b57f78cf Revert extreme settings 2022-04-20 13:59:12 +02:00
alesapin
5334522d5c Extreme settings 2022-04-20 13:33:42 +02:00
alesapin
2eee79dc2a Review fixes 2022-04-20 13:26:20 +02:00
alesapin
40c15222f8 Merge branch 'master' into fix_trash 2022-04-20 12:45:49 +02:00
Antonio Andelic
d77678d1d1
Merge pull request #36410 from xinhuitian/move_down_data_range_construction
Move down data_range construction in filterMarksUsingIndex and filterMarksUsingMergedIndex
2022-04-20 12:18:05 +02:00
tavplubix
2c54381520
Merge pull request #36200 from ClickHouse/fix_create_as_replicated_default_args
Fix CREATE AS for ReplicatedMergeTree with default arguments
2022-04-20 12:01:13 +03:00
Robert Schulze
b24ca8de52
Fix various clang-tidy warnings
When I tried to add cool new clang-tidy 14 warnings, I noticed that the
current clang-tidy settings already produce a ton of warnings. This
commit addresses many of these. Almost all of them were non-critical,
i.e. C vs. C++ style casts.
2022-04-20 10:29:05 +02:00
Kseniia Sumarokova
766a84e64f
Merge pull request #35915 from kssenii/better-s3-settings
Better settings configuration for s3
2022-04-20 10:22:06 +02:00
Robert Schulze
b6d7367538
Merge remote-tracking branch 'origin/master' into erase_if3
Conflicts:
- Interpreters/ActionsDAG.cpp
2022-04-20 10:02:59 +02:00
Antonio Andelic
f0774ca9ee
Merge pull request #36393 from tcoyvwac/fix/prefer-standard-algorithms-std-transform-storages
Minor refactor to prefer C++ Standard Algorithms
2022-04-20 08:42:54 +02:00
Antonio Andelic
bbb0be6a44
Merge pull request #36402 from rschu1ze/clang-tidy-contains
Activate clang-tidy warning "readability-container-contains"
2022-04-20 08:15:46 +02:00
xiedeyantu
47c826935d temporary table can show total rows and total bytes 2022-04-20 10:03:16 +08:00
avogar
f31f019252 Fix 2022-04-19 19:25:41 +00:00
avogar
1f252cedfe Make better 2022-04-19 19:16:47 +00:00
avogar
9a81b67ba5 Fix bug in s3Cluster schema inference 2022-04-19 18:23:04 +00:00
Anton Popov
bee4ca9b62 add more tests for error diagnostics in files 2022-04-19 15:56:34 +00:00
alesapin
aea7c7755a Fix style 2022-04-19 17:25:08 +02:00
Anton Popov
3e361c9759 Merge remote-tracking branch 'upstream/master' into HEAD 2022-04-19 14:18:04 +00:00
alesapin
7919485db2 Fix 2022-04-19 15:57:25 +02:00
alesapin
7cb7c120cc Less ugly 2022-04-19 15:53:10 +02:00
Anton Popov
056500f2ad
Merge pull request #33391 from zhongyuankai/fix_storage_distributed_ttl
Fix Alter ttl modification unsupported table engine
2022-04-19 15:47:01 +02:00
alesapin
cc06bc3d99 Add some clarifications 2022-04-19 14:01:30 +02:00
tcoyvwac
2441d33940
storages: prefer standard algorithms
StorageSystemZooKeeper.cpp:
Prefer std::any_of over ranged-for.
2022-04-19 13:40:50 +02:00
tcoyvwac
f7789c9b6d
storages: prefer standard algorithms
StorageSystemZooKeeper.cpp:
Prefer std::transform over ranged-for.
2022-04-19 13:40:49 +02:00
tcoyvwac
10dfeedd23
storages: prefer standard algorithms
StorageSystemZooKeeper.cpp:
Prefer std::transform over ranged-for.
2022-04-19 13:40:49 +02:00
alesapin
ee8d26ff0e Better test and fix for move 2022-04-19 13:27:55 +02:00
Sergei Trifonov
75c1053ad7 Merge remote-tracking branch 'origin/master' into surahman-CH-31221-AWS-S3-object-version-id 2022-04-19 13:17:25 +02:00
alesapin
8ade9fba52 Fix 2022-04-19 11:56:14 +02:00
mergify[bot]
d894aed7ae
Merge branch 'master' into mongo-fix 2022-04-19 09:32:51 +00:00
kssenii
0e05e1fa2f Fix 2022-04-19 11:31:27 +02:00
kssenii
1f9a597b77 Merge master 2022-04-19 11:31:13 +02:00
kssenii
f8f66dd23d Better 2022-04-19 10:59:47 +02:00
xinhuitian
5d0957e669 move down data range construction in filterMarksUsingIndex and filterMarksUsingMergedIndex 2022-04-19 13:26:33 +08:00
alesapin
c81cb9e563 Better 2022-04-19 01:12:07 +02:00
alesapin
bd7b3847c1 Some code 2022-04-19 01:09:09 +02:00
Robert Schulze
118e94523c
Activate clang-tidy warning "readability-container-contains"
This check suggests replacing <Container>.count() by
<Container>.contains() which is more speaking and in case of
multimaps/multisets also faster.
2022-04-18 23:53:11 +02:00
mergify[bot]
daca157111
Merge branch 'master' into fix_storage_distributed_ttl 2022-04-18 17:31:59 +00:00
Memo
a17f102fe6 fix style 2022-04-18 20:31:40 +08:00
alesapin
a97754f462 Fix 2022-04-18 13:39:09 +02:00
alesapin
c4b70f3ed3 Merge branch 'master' into fix_trash 2022-04-18 12:20:09 +02:00
zhanglistar
5670dcf750 Fix crash in debug mode of ReadBufferFromHDFS. 2022-04-18 17:52:59 +08:00
zhanglistar
355333b361
Merge branch 'ClickHouse:master' into fixhdfsreadbuffer 2022-04-18 16:42:03 +08:00
Memo
b3adf150b5 add_query_level_settings 2022-04-18 12:15:41 +08:00
Alexey Milovidov
f6ab2bd523
Merge pull request #36312 from ClickHouse/remove-arcadia
Remove remaining parts of Arcadia
2022-04-18 07:02:54 +03:00
Alexey Milovidov
36595e4206
Merge pull request #36320 from ClickHouse/fix-clang-tidy-14
Fix clang-tidy-14 (part 1)
2022-04-18 07:02:10 +03:00
Alexey Milovidov
242919eddd Remove abbreviation 2022-04-18 01:02:49 +02:00
Robert Schulze
1e1df8e101
Replace remove-erase idiom by C++20 erase()/erase_if()
- makes the code less verbose while being as efficient
2022-04-17 12:04:47 +02:00
Alexey Milovidov
294efeccfe Fix clang-tidy-14 (part 1) 2022-04-16 04:54:04 +02:00
Anton Popov
2de6668b3f show names of erroneous files 2022-04-16 00:10:47 +00:00
Alexey Milovidov
cbeeb7ec4f Remove Arcadia 2022-04-16 00:20:47 +02:00
alesapin
1706ae9e15 Some trash implementation 2022-04-15 18:36:23 +02:00
alesapin
5a8419a48e Remove more trash 2022-04-15 17:05:17 +02:00
alesapin
eb7593f786 Remove more trash 2022-04-15 16:24:38 +02:00
zhanglistar
fe93fd83f5
Merge branch 'ClickHouse:master' into fixhdfsreadbuffer 2022-04-15 16:27:20 +08:00
zhanglistar
e22a175afb Fix ReadBufferFromHDFS crash in debug mode 2022-04-15 14:04:30 +08:00
Anton Popov
305dd57262
Merge branch 'master' into fix_storage_distributed_ttl 2022-04-14 14:51:15 +02:00
tavplubix
3af2384066
Merge pull request #36190 from azat/revert-27002-mutation-stuck
Revert "Fix possible mutation stuck due to race with DROP_RANGE"
2022-04-14 15:03:35 +03:00
alesapin
bd629f4cd0
Merge pull request #36163 from ClickHouse/maybe_fix_36098
Some fixes for ReplicatedMergeTree
2022-04-14 12:12:55 +02:00
tavplubix
fd23c42ea7
Update StorageReplicatedMergeTree.cpp 2022-04-13 22:33:55 +03:00
avogar
42726639f3 Check ORC/Parquet/Arrow format magic bytes before loading file in memory 2022-04-13 19:27:38 +00:00
avogar
1c065f8c7a Some refactoring around schema inference with globs 2022-04-13 17:02:48 +00:00
Alexander Tokmakov
66fdf35dfd remove outdated parts immediately on drop partition 2022-04-13 18:01:22 +02:00
Alexander Gololobov
fcb83a12ff
Merge pull request #35953 from azat/ttl-mutations
Require mutations for per-table TTL only when it had been changed
2022-04-13 17:14:04 +02:00
Alexander Tokmakov
40fea2b833 do not unfold {uuid} macro 2022-04-13 16:51:59 +02:00
kssenii
0912b038fa Merge master 2022-04-13 15:30:01 +02:00
Azat Khuzhin
6dbc138e76 Remove duplicated parts removal from mutation.
This commit reverts 701e2ffd79 ("Fix
possible mutation stuck due to race with DROP_RANGE"), and also adds
some clarification comments.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-13 16:09:17 +03:00
Sergei Trifonov
9d1a2da685 merge master and fix conflicts 2022-04-13 14:53:36 +02:00
Mikhail Artemenko
2fd86cc564
Merge branch 'master' into MeiliSearch 2022-04-13 12:05:46 +03:00
Mikhail Artemenko
4ed6ec6780 change fast_test references 2022-04-13 10:35:37 +03:00
Julian Gilyadov
a4f56f3330
Throw exception when file cant be executed instead of displaying success 2022-04-12 17:52:44 +02:00
mergify[bot]
0b3c15c07a
Merge branch 'master' into projection-fix1 2022-04-12 13:49:28 +00:00
Alexander Tokmakov
457a9e9691 fixes for ReplicatedMergeTree 2022-04-12 14:14:26 +02:00
Vladimir C
8873d7b0e7
Merge pull request #36082 from bigo-sg/cache_hive_files 2022-04-12 11:42:08 +02:00
Kseniia Sumarokova
b29202ac88
Merge pull request #36150 from kssenii/fix-cache-test
Fix test 02241_remote_filesystem_cache_on_insert for database Ordinary
2022-04-12 11:08:44 +02:00
taiyang-li
9d31c44d39 fix bug of read buffer from hdfs 2022-04-12 12:01:51 +08:00
Kseniia Sumarokova
51dd39b42f
Update StorageSystemRemoteDataPaths.cpp 2022-04-11 22:46:51 +02:00
kssenii
fc62e95d76 Fix 2022-04-11 22:24:45 +02:00
Kruglov Pavel
972357ebf6
Merge pull request #35748 from bigo-sg/s3_cluster_join
Fixed : elimate AMBIGUOUS_COLUMN_NAME exception in hdfs/s3 cluster join query
2022-04-11 16:22:53 +02:00
Nikolai Kochetov
ac74757f92
Merge pull request #35973 from filimonov/kafka_read_consumers_in_parallel
Make parallel reading from several kafka consumers work again
2022-04-11 15:30:52 +02:00
Kseniia Sumarokova
beea4b103f
Merge pull request #36109 from ucasfl/window-view-fix
Fix window view when is proc time and window kind larger than day
2022-04-11 09:54:14 +02:00
taiyang-li
090fd72884 fix bug 2022-04-11 11:19:31 +08:00
Denny Crane
383076ca0e
fix typo 2022-04-10 16:42:43 -03:00
fenglv
cac2df914b fix 2022-04-10 17:28:17 +00:00
fenglv
24e86f81cb Fix window view when is proc time and window kind larger than day
fix
2022-04-10 17:26:09 +00:00
taiyang-li
7e89f760f3 remove useless code 2022-04-09 10:43:58 +08:00
tavplubix
faae540a85
Merge pull request #24258 from ClickHouse/mvcc_prototype
MVCC prototype for MergeTree
2022-04-08 22:02:13 +03:00
Kseniia Sumarokova
1e54c5c57b
Merge pull request #35885 from kssenii/fix-postgres-test-2
fix postgres test
2022-04-08 19:55:05 +02:00
taiyang-li
70f4503ba5 use global context for cache 2022-04-09 00:28:07 +08:00
taiyang-li
cd807da838 finish test 2022-04-09 00:15:33 +08:00
taiyang-li
e319df1799 finish dev 2022-04-08 23:58:56 +08:00
taiyang-li
2c99ef0ecc refactor HiveTableMetadata 2022-04-08 23:04:24 +08:00
Alexander Tokmakov
141fbc6980 Merge branch 'master' into mvcc_prototype 2022-04-08 13:38:11 +02:00
Alexander Tokmakov
49c35f3261 Merge branch 'master' into mvcc_prototype 2022-04-08 13:34:40 +02:00
Anton Popov
a16ac67c0c
Merge pull request #36034 from amosbird/better-projection2
Use storage_snapshot for projection analysis
2022-04-08 13:22:53 +02:00
Vitaly Baranov
45dea9933b
Merge pull request #35883 from azat/format-settings
Do not allow SETTINGS after FORMAT for INSERT queries
2022-04-08 12:38:14 +02:00
kssenii
87855c4ae7 Merge master 2022-04-08 12:32:48 +02:00
kssenii
ef2b238444 Fix 2022-04-08 12:30:24 +02:00
alesapin
8ec802bc62
Merge pull request #35475 from kssenii/remote-fs-cache-improvements
Allow to write remote fs cache on all write operations. Add `system.remote_filesystem_cache` table. Add `drop remote filesystem cache (<path>)` query. Add `system.remote_data_paths` table.
2022-04-08 12:06:26 +02:00
taiyang-li
2e6f0db825 first commit 2022-04-08 15:12:24 +08:00
taiyang-li
dc4c638af8 Merge branch 'master' into optimize_trivial_hive_query 2022-04-08 11:36:55 +08:00
lgbo-ustc
759e4a9a05 Merge remote-tracking branch 'ck/master' into s3_cluster_join 2022-04-08 09:25:30 +08:00
Alexander Tokmakov
6a46da93ae Merge branch 'master' into mvcc_prototype 2022-04-07 23:22:19 +02:00
kssenii
5dce2f18b5 Better 2022-04-07 18:48:35 +02:00
Kseniia Sumarokova
5f9bc1c68e
Merge pull request #35987 from awakeljw/fork_chmaster
[Improvement] improvement in PARTITION ALL
2022-04-07 17:36:35 +02:00
Sergei Trifonov
01425df758 fix build 2022-04-07 16:07:12 +02:00
Azat Khuzhin
33d99c8ffb Introduce compatiblity mode for SETTINGS after FORMAT
Add allow_settings_after_format_in_insert setting, OFF by default.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
v2: s/parser_settings_after_format_compact/allow_settings_after_format_in_insert/ (suggested by vitlibar)
v3: replace ParserSettings with a flag (requested by vitlibar)
2022-04-07 16:36:34 +03:00
Alexander Tokmakov
7f54e7b422 Merge branch 'master' into mvcc_prototype 2022-04-07 15:14:06 +02:00
taiyang-li
87507ec9e8 fix conflicts 2022-04-07 20:52:54 +08:00
taiyang-li
d7c79c3a54 merge master and solve conflicts 2022-04-07 20:48:16 +08:00
taiyang-li
e9de38c52b fix bug 2022-04-07 20:45:07 +08:00
Amos Bird
a071bde59e
Use storage_snapshot for projection analysis 2022-04-07 19:56:31 +08:00
Alexander Tokmakov
8290ffa88d Merge branch 'master' into mvcc_prototype 2022-04-07 13:50:42 +02:00
Nikolai Kochetov
a998d73ee2
Update StorageKafka.cpp 2022-04-07 12:37:11 +02:00
Kseniia Sumarokova
5909b3c04d
Merge pull request #36015 from ucasfl/filelog-fix
Fix data race in StorgeFileLog
2022-04-07 12:23:12 +02:00
Sergei Trifonov
755d5c55f9 resolve conflict 2022-04-07 11:57:45 +02:00
Amos Bird
1238babb6f
Make SelectQueryInfo pseudo-copyable 2022-04-07 17:46:50 +08:00
Amos Bird
9cf5935604
Fix broken SET reuse during projection analysis. 2022-04-07 17:46:49 +08:00
jewisliu
77edd41b2e [Improvement] improvement in PARTITION ALL
1. ASTPartition::formatImpl should output ALL while executing ALTER TABLE t DETACH PARTITION ALL
2. prohibit PARTITION ALL excepte DETACH PARTITION ALL
2022-04-07 17:37:01 +08:00
Sergei Trifonov
9c2804b924 merge from master + resolve conflicts 2022-04-07 11:22:01 +02:00
Mikhail Artemenko
0f1119dd2a fix impl after merge 2022-04-07 11:56:30 +03:00
taiyang-li
2dc420c66b rename some symbols in hivefile 2022-04-07 15:48:42 +08:00
taiyang-li
4763a39802 merge bigo-sg/use_minmax_index and solve conflict 2022-04-07 15:45:28 +08:00
taiyang-li
046a2ba51c rename some symboles 2022-04-07 15:35:08 +08:00
taiyang-li
ad074fee91 merge use_minmax_index and solve conflict 2022-04-07 15:19:45 +08:00
taiyang-li
f02d769343 fix build error 2022-04-07 14:29:35 +08:00
taiyang-li
acc7046d54 remove some useless virtual and rename some functions in HiveFile 2022-04-07 11:46:57 +08:00
taiyang-li
df00bd214d merge bigo-sg/use_minmax_index and solve conflict 2022-04-07 11:18:24 +08:00
taiyang-li
2ef316801c Merge branch 'master' into use_minmax_index 2022-04-07 10:53:25 +08:00
fenglv
517c2ae8df Fix data race in StorgeFileLog 2022-04-07 02:20:20 +00:00
tavplubix
58db8cd9b3
Merge pull request #35946 from ClickHouse/faster_replicated_ddl
Make some replicated DDL queries faster
2022-04-06 19:25:25 +03:00
Mikhail Artemenko
151eeb1a27
Merge branch 'master' into MeiliSearch 2022-04-06 17:07:55 +03:00
Kseniia Sumarokova
2aa3d32887
Merge pull request #35980 from ucasfl/thread_num
parallel reading files for FileLog Engine
2022-04-06 15:05:41 +02:00
Antonio Andelic
cda8a18fe3
Merge pull request #35984 from ClickHouse/fix-url-data-race
Fix data race in StorageURL
2022-04-06 14:36:19 +02:00
kssenii
846faa51d8 Fix bug 2022-04-06 14:35:19 +02:00
Alexander Tokmakov
1309e781b6 apply suggestion 2022-04-06 13:56:26 +02:00
taiyang-li
0b0c8ef09e add integration tests 2022-04-06 18:47:34 +08:00
Kseniia Sumarokova
a383d9ee6a
Merge pull request #35916 from filimonov/kafka_metrics
Add some metrics to engine Kafka
2022-04-06 11:58:17 +02:00
Kseniia Sumarokova
bb26903675
Merge pull request #35771 from bigo-sg/fix-filebuffer-pos
fix filebuffer pos in RemoteReadBuffer
2022-04-06 11:57:05 +02:00
taiyang-li
acb9f1632e suppoort skip splits in orc and parquet 2022-04-06 16:40:22 +08:00
Antonio Andelic
84eef61d17 Pull under reader mutex 2022-04-06 06:39:56 +00:00
fenglv
a2ce366c34 parallel reading files for FileLog Engine 2022-04-06 04:49:43 +00:00
taiyang-li
43e8af697a fix code style 2022-04-06 11:41:16 +08:00
Mikhail Filimonov
f9a4cd12d4
Make parallel reading of kafka consumers work again (issue #35153) 2022-04-05 21:46:56 +02:00
Mikhail Filimonov
3412be9d4d
Change KafkaDirectReads to KafkaConsumersInUse 2022-04-05 20:25:05 +02:00
Nikolai Kochetov
def33323a0
Merge pull request #35788 from amosbird/better-projection1
Improve minmax_count_projection
2022-04-05 19:59:45 +02:00
kssenii
cbd7f4acfb Fix 2022-04-05 19:24:10 +02:00
Alexander Tokmakov
37a06eec1a fixes 2022-04-05 17:36:53 +02:00
Alexander Tokmakov
d9e3e9b69f Merge branch 'master' into faster_replicated_ddl 2022-04-05 14:38:38 +02:00
Alexander Tokmakov
1fe50ad201 Merge branch 'master' into mvcc_prototype 2022-04-05 14:38:02 +02:00
Azat Khuzhin
8ebaf84989 Fix clang-tidy readability-simplify-boolean-expr warning in AlterCommands.cpp
I want to make the code more readable before, but clang-tidy is too
smart...

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-05 13:39:39 +03:00
alesapin
eaa72ef002
Merge pull request #35904 from ClickHouse/more_logs_on_unsuccessful_part_removal
More logs on unsuccessful part removal
2022-04-05 11:29:08 +02:00
Alexander Tokmakov
da00beaf7f Merge branch 'master' into mvcc_prototype 2022-04-05 11:14:42 +02:00
Azat Khuzhin
c6f182163a Require mutations for per-table TTL only when it had been changed
Before this patch only per-column TTL did not requires mutation if it
had not been changed, after per-table TTL will also check whether it had
been changed or not.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-05 11:08:35 +03:00
Amos Bird
5bc09550d8
Fix tests 2022-04-05 15:56:39 +08:00
Amos Bird
35a8bb2a9b
add comment 2022-04-05 15:56:38 +08:00
Amos Bird
163664fad7
Improve minmax_count_projection 2022-04-05 15:56:37 +08:00
Vladimir C
54b4049ea5
Merge pull request #35794 from awakeljw/fork_chmaster 2022-04-05 09:44:45 +02:00
Alexander Tokmakov
4e9ec5dc2f make some replicated DDL faster 2022-04-05 00:51:48 +02:00
Alexey Milovidov
1b562923f9
Merge pull request #35941 from ClickHouse/revert-35184-clang-tidy-issues
Revert "clang-tidy report issues with Medium priority"
2022-04-04 23:42:21 +03:00
alesapin
69fceb7e6d
Merge pull request #35926 from ClickHouse/better_consumers
Better check for kafka_num_consumers
2022-04-04 22:42:05 +02:00
Alexey Milovidov
4d6c030d23
Revert "clang-tidy report issues with Medium priority" 2022-04-04 23:41:42 +03:00
Mikhail Artemenko
49236c691e replace curl with Poco::Net 2022-04-04 23:27:39 +03:00
Mikhail Filimonov
53c7376e37
Add some metrics to engine Kafka 2022-04-04 20:25:53 +02:00
alesapin
d04c48e67a Fix build 2022-04-04 20:14:09 +02:00
alesapin
1e4fe5e3b5
Merge pull request #35882 from ClickHouse/more_metadata_alters
Make more alters of nested types metadata-only
2022-04-04 20:11:31 +02:00
alesapin
0477e74f42 Get rid of caps 2022-04-04 19:41:54 +02:00
alesapin
72331856eb fix message 2022-04-04 18:28:26 +02:00
alesapin
86f42e7a3a Better check for kafka_num_consumers 2022-04-04 18:07:36 +02:00
Anton Popov
bd93be65aa
Merge pull request #35820 from azat/columns-ttl-fix
Avoid processing per-column TTL multiple times
2022-04-04 16:42:50 +02:00
Anton Popov
d08d4a2437
Merge pull request #35806 from CurtizJ/dynamic-columns-8
Fix inserts to columns of type `Object` in partitioned tables
2022-04-04 16:16:54 +02:00
kssenii
4e50da722b Better s3 settings 2022-04-04 16:14:56 +02:00
Antonio Andelic
db75bf6f5d
Merge pull request #35852 from ClickHouse/bigo-sg-add_hints
Refactoring of hints for column descriptor
2022-04-04 15:37:08 +02:00
alesapin
3ccf99c3d7
Merge pull request #35884 from ClickHouse/don_t_allow_to_reset_settings
Forbid to reset non existing settings
2022-04-04 15:27:34 +02:00
Alexander Tokmakov
a2167f12b8 Merge branch 'master' into mvcc_prototype 2022-04-04 14:24:23 +02:00
Nikita Taranov
bd89fcafdb
Make SortDescription::column_name always non-empty (#35805) 2022-04-04 14:17:15 +02:00
alesapin
9b75ef6ce9 Fix build 2022-04-04 14:03:16 +02:00
alesapin
885447342c More logs on unsuccessful part removal 2022-04-04 13:17:33 +02:00
alesapin
803a1a2a9c Fix tests and check 2022-04-04 11:19:02 +02:00
Anton Popov
ae53aae106 fix clang-tidy 2022-04-04 08:48:31 +00:00
taiyang-li
38f149b533 optimize trivial count hive query 2022-04-04 15:28:26 +08:00
Antonio Andelic
a926bc19ea Address PR comments 2022-04-04 07:24:42 +00:00
Antonio Andelic
d96b682a55 Refactor 2022-04-04 07:24:42 +00:00
taiyang-li
6bc1786047 fix style 2022-04-04 07:24:42 +00:00
taiyang-li
d6247338de fix failed stateless tests 2022-04-04 07:24:42 +00:00
taiyang-li
fd9a10ef53 add hints for projections 2022-04-04 07:24:42 +00:00
taiyang-li
4547ed370a add hints for column description 2022-04-04 07:24:42 +00:00
taiyang-li
4e2d5f1841 Merge remote-tracking branch 'bigo-sg/use_minmax_index' into optimize_trivial_hive_query 2022-04-04 10:42:28 +08:00
taiyang-li
cbfc0f6bac fix typo 2022-04-04 10:42:22 +08:00
alesapin
a8c1ccdb20 Forbit to reset non existing settings 2022-04-03 18:29:24 +02:00
Kseniia Sumarokova
30c1afe29d
Merge pull request #35646 from bigo-sg/reduce-mutex-scope
narrow mutex scope when setenv LIBHDFS3_CONF
2022-04-03 17:59:24 +02:00
alesapin
7b35920d4c Make more alters of nested types metadata-only 2022-04-03 14:03:34 +02:00
alesapin
91453fe4d6
Merge pull request #35759 from chen9t/fix_dirty_metacache
Clean dirty meta cache when part is broken during part loading phase
2022-04-03 13:33:46 +02:00
Azat Khuzhin
195b4c47ea Avoid processing per-column TTL multiple times
Before this patch ttl.txt will not be written for per-column TTLs, and
hence it will be calculated every time after server restart of
DETACH/ATTACH cycle (note, that it will work w/o restart since in-memory
representation will avoid this).

v2: convert test to .sh to get correct current database over default for MV
v3: extract UUID to avoid error like in [1]:
    [ 490 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) []: Code: 349. DB::Exception: Cannot convert NULL value to non-Nullable type: While processing query_id LIKE concat('%', CAST(_CAST(NULL, 'Nullable(UUID)') AS uuid, 'String'), '%'): while pushing to view test_0hc2ro.this_text_log (c64e5af4-059e-4330-a728-354ecf83c031). (CANNOT_INSERT_NULL_IN_ORDINARY_COLUMN)
    [1]: https://s3.amazonaws.com/clickhouse-test-reports/35820/a512d322b024d37d2f1082c4833f59f86057555f/stateless_tests_flaky_check__address__actions_.html
v4: add no-parallel to avoid issues with disappeared underlying table while pushing to text_log
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-02 15:19:39 +03:00
shuchaome
a565a93740 reduce mutex scope when setenv LIBHDFS3_CONF 2022-04-02 06:52:39 +08:00
kssenii
6c8e073a61 Merge master 2022-04-01 16:48:29 +02:00
Alexander Tokmakov
6591d1ceb7 Merge branch 'master' into mvcc_prototype 2022-04-01 15:38:46 +02:00
fenglv
97523a3329 Try remove unneed variable 2022-04-01 09:58:23 +00:00
Kseniia Sumarokova
d3b3294872
Merge pull request #35365 from bigo-sg/improve_access_type
Improve check access in table functions
2022-04-01 10:47:02 +02:00
Kseniia Sumarokova
9c39320e75
Merge pull request #35800 from ucasfl/mutex
remove unused variable
2022-04-01 10:46:23 +02:00
Kseniia Sumarokova
04729ecc70
Merge pull request #35798 from xiedeyantu/patch-1
Delete duplicate code
2022-04-01 10:43:31 +02:00
jewisliu
9a4cc78dfb support ALTER TABLE t DETACH PARTITION ALL syntax 2022-04-01 13:13:07 +08:00
Alexey Milovidov
5c6b84c398
Merge pull request #35586 from kssenii/fix-virtual-columns-s3
Fix s3 engine getting virtual columns
2022-04-01 04:43:32 +03:00
Alexander Tokmakov
63e8d73ba4 better message for incorrect *MergeTree arguments 2022-03-31 17:08:30 +02:00
Anton Popov
42acb1dc29 fix inserts to columns of type Object in partitioned tables 2022-03-31 13:30:01 +00:00
fenglv
e2e4b02d13 remove unused variable 2022-03-31 12:01:10 +00:00
xiedeyantu
d141dbc388
Delete duplicate code
Delete duplicate code
2022-03-31 19:34:30 +08:00
Nikita Mikhaylov
f6bfdcc0c9
Added support for schema inference for hdfsCluster (#35602) 2022-03-31 12:47:36 +02:00
Alexander Tokmakov
5a50ad9de3 Merge branch 'master' into mvcc_prototype 2022-03-31 11:35:04 +02:00
Kseniia Sumarokova
a711fc0402
Merge pull request #35572 from bigo-sg/hdfs_config_prefix
Deduce absolute hdfs config path
2022-03-31 10:08:32 +02:00
Antonio Andelic
13b4fabee4
Merge pull request #35743 from bigo-sg/remote_filter_for_hive
Respect remote_url_allow_hosts for hive
2022-03-31 08:54:37 +02:00
taiyang-li
539ce5f1b8 Merge branch 'master' into use_minmax_index 2022-03-31 12:27:32 +08:00
taiyang-li
70087bc959 Merge branch 'master' into improve_access_type 2022-03-31 12:19:17 +08:00
taiyang-li
f6ccda8097 Merge branch 'master' into hdfs_config_prefix 2022-03-31 12:16:05 +08:00
chen9t
b40a69f59a Code style 2022-03-31 10:10:05 +08:00
kssenii
98e85dd68e Fix build 2022-03-30 20:27:04 +02:00
shuchaome
738966b6b6 fix filebuffer pos in RemoteReadBuffer 2022-03-31 00:03:45 +08:00
kssenii
099fa0e668 Merge master 2022-03-30 17:33:32 +02:00
kssenii
0fc92fe2aa Get rid of all "remote" mentions 2022-03-30 17:15:29 +02:00
chen9t
5cb2301e39 Clean dirty meta cache when part is broken during part loading phase 2022-03-30 20:40:20 +08:00
Antonio Andelic
a3ad99be03 Create CallbackRunner in Storage 2022-03-30 11:24:50 +00:00
kssenii
98ad3f4a91 Fix tests, rename some remote mentions 2022-03-30 12:21:59 +02:00
lgbo-ustc
08e1fdc6b8 Fixed :a elimate AMBIGUOUS_COLUMN_NAME exception
elimate AMBIGUOUS_COLUMN_NAME exception in hdfs/s3 cluster when run a query with join
2022-03-30 17:06:38 +08:00
Antonio Andelic
1173eab15e Use same IOThreadPool for URL and S3 2022-03-30 08:15:20 +00:00
taiyang-li
16bb4c4ad0 respect remote_url_allow_hosts for hive 2022-03-30 15:33:59 +08:00
Antonio Andelic
03a09dc914
Merge pull request #35571 from ClickHouse/parallel-s3-downloading
Parallel S3 downloading
2022-03-30 08:14:49 +02:00
Antonio Andelic
d85ed8f2a9
Merge pull request #35655 from ClickHouse/exception-compile-time-message-check
Use compile-time check for `Exception` messages
2022-03-30 08:11:32 +02:00
taiyang-li
0af6fdb576 fix building 2022-03-30 11:28:21 +08:00
taiyang-li
fcfabf4a0c Merge remote-tracking branch 'ck/master' into use_minmax_index 2022-03-30 11:23:24 +08:00
taiyang-li
e76ed3acde Merge branch 'master' into improve_access_type 2022-03-30 11:04:57 +08:00
taiyang-li
9ffb42d507 Merge branch 'master' into hdfs_config_prefix 2022-03-30 11:02:31 +08:00
Anton Popov
1cba31c305
Merge pull request #35687 from CurtizJ/dynamic-columns-7
Fix reading from type object
2022-03-30 00:21:32 +02:00
Alexander Tokmakov
287d858fda Merge branch 'master' into mvcc_prototype 2022-03-29 16:24:12 +02:00
Dmitry Novik
8f935a72d6
Merge pull request #33230 from CurtizJ/read_in_order_max_rows_to_read
Proper handle of 'max_rows_to_read' in case of reading in order of sorting key
2022-03-29 15:16:34 +02:00
Antonio Andelic
ef484547bf Fix GCC build 2022-03-29 13:16:11 +00:00
taiyang-li
67c3c0be3d Merge branch 'master' into improve_access_type 2022-03-29 20:42:57 +08:00
taiyang-li
631352ba4b Merge branch 'master' into hdfs_config_prefix 2022-03-29 20:40:32 +08:00
kssenii
4374415bd7 Fix clang-tidy, style check 2022-03-29 14:20:21 +02:00
Kruglov Pavel
fe60560c03
Merge pull request #35619 from Avogar/check-remote-url-allow-hosts
Check remote_url_allow_hosts before schema inference in URL engine
2022-03-29 14:08:02 +02:00
Anton Popov
d42632dc13
Merge pull request #32928 from bigo-sg/rocksdb_metacache
Speed up parts loading process of MergeTree by metadata cache
2022-03-29 14:05:22 +02:00
taiyang-li
59727a423e Merge remote-tracking branch 'ck/master' into use_minmax_index 2022-03-29 19:39:09 +08:00
Maksim Kita
a1a4552740
Merge pull request #35184 from DevTeamBK/clang-tidy-issues
clang-tidy report issues with Medium priority
2022-03-29 13:19:54 +02:00
taiyang-li
77314246f3 Merge branch 'master' into hdfs_config_prefix 2022-03-29 18:32:59 +08:00
taiyang-li
834ac3a1b9 Merge remote-tracking branch 'ck/master' into rocksdb_metacache 2022-03-29 18:32:06 +08:00
Nikolai Kochetov
86690f18ee
Merge pull request #35667 from amosbird/projection-fix2
Another fix of projection part check
2022-03-29 10:22:01 +02:00
Antonio Andelic
362e877529 Merge branch 'master' into parallel-s3-downloading 2022-03-29 07:05:26 +00:00
Amos Bird
3ab8fc1e7b
Fix projection part check
Fix projection part check when projection part type is different from
parent part type.
2022-03-29 10:39:07 +08:00
kssenii
44157e5c80 Merge master 2022-03-29 02:19:32 +02:00
kssenii
c6b1f86306 Merge with master 2022-03-29 02:03:28 +02:00
Nikita Mikhaylov
85282f1b4d
Made schema inference work with s3Cluster table function (#35544) 2022-03-29 00:46:35 +02:00
kssenii
1d49a85963 Merge master 2022-03-28 22:51:56 +02:00
kssenii
a2013ebe2b Better 2022-03-28 22:40:27 +02:00
Kseniia Sumarokova
9a8e9198e5
Merge pull request #35593 from kssenii/named-collections-fix
Fix headers with named collections, add compression_method
2022-03-28 22:10:32 +02:00
Alexander Tokmakov
208b242188 Merge branch 'master' into mvcc_prototype 2022-03-28 19:58:06 +02:00
Anton Popov
0e4af89f69 fix reading from type object 2022-03-28 17:23:34 +00:00
mergify[bot]
f09ebea2d8
Merge branch 'master' into read_in_order_max_rows_to_read 2022-03-28 13:30:34 +00:00
taiyang-li
38702be40c Merge remote-tracking branch 'ck/master' into rocksdb_metacache 2022-03-28 21:03:22 +08:00
mergify[bot]
70b339ea4e
Merge branch 'master' into check-remote-url-allow-hosts 2022-03-28 11:55:48 +00:00
mergify[bot]
a022de4a6e
Merge branch 'master' into fix-materialized-postgresql 2022-03-28 10:55:41 +00:00
Antonio Andelic
d9d826c813 Address PR review 2022-03-28 08:19:23 +00:00
kssenii
c4832f7633 Fix 2022-03-28 09:28:02 +02:00
Mikhail Artemenko
1886d9c2ea refactor parseTypeOfField & fix tests 2022-03-27 23:00:28 +03:00
kssenii
c6706b8bfa Better 2022-03-25 19:30:13 +01:00
kssenii
141c290344 Fix build 2022-03-25 18:31:15 +01:00
taiyang-li
b79cec6806 Merge branch 'use_minmax_index' of https://github.com/bigo-sg/ClickHouse into use_minmax_index 2022-03-25 23:33:49 +08:00
taiyang-li
eee8949150 fix code 2022-03-25 23:33:46 +08:00
Alexander Tokmakov
9ec0951de5 update comment for mismatching checksums 2022-03-25 15:54:05 +01:00
taiyang-li
4aaa361f2e Merge remote-tracking branch 'ck/master' into use_minmax_index 2022-03-25 22:48:03 +08:00
taiyang-li
fbadd8fc0e only deduce absolute path of hdfs config when it's relative path do not exists 2022-03-25 22:28:47 +08:00
taiyang-li
c013cd0b52 Merge branch 'master' into hdfs_config_prefix 2022-03-25 22:13:45 +08:00
taiyang-li
3ca487030d Merge branch 'master' into improve_access_type 2022-03-25 22:11:08 +08:00
avogar
ecc57b436c Check remote_url_allow_hosts before schema inference in URL engine 2022-03-25 13:30:45 +00:00
Antonio Andelic
394682b983 Merge branch 'master' into parallel-s3-downloading 2022-03-25 13:00:44 +00:00
kssenii
6769b84638 Merge master 2022-03-24 20:47:43 +01:00
kssenii
d9d9f3bc6d Fix 2022-03-24 19:10:53 +01:00
kssenii
e91e30aaa4 Fix 2022-03-24 17:10:27 +01:00
Sergei Trifonov
a90e83665d
Merge pull request #35343 from ClickHouse/parallel-multipart-upload-for-s3storage
allow parallelization of multipart S3 storage upload; fixes #34350
2022-03-24 15:58:29 +01:00
kssenii
d6ab6de30a Minor changes 2022-03-24 15:32:08 +01:00
taiyang-li
c0eb9f73a9 remote std::cout 2022-03-24 21:17:43 +08:00
taiyang-li
6eccd09e2b deduce hdfs config path 2022-03-24 18:04:29 +08:00
Antonio Andelic
62a495a5fc Small refactor 2022-03-24 09:45:30 +00:00
Nikolai Kochetov
283e20a9a5
Merge pull request #35395 from amosbird/distributedmultiplejoin
Validate some thoughts over making sets
2022-03-24 10:30:26 +01:00
Antonio Andelic
c52cee2219 Merge branch 'master' into parallel-s3-downloading 2022-03-24 08:11:52 +00:00
Maksim Kita
1df1721648
Merge pull request #35546 from kitaisreal/refactor-external-models-loader
ExternalModelsLoader refactoring
2022-03-24 01:24:45 +01:00
kssenii
d4161b5925 Add optin read_from_cache_if_exists_otherwise_bypass_cache (for merges) 2022-03-23 20:24:00 +01:00
Maksim Kita
3df6a86387 ExternalModelsLoader refactoring 2022-03-23 20:05:17 +01:00
kssenii
d2a3cfe5dc Cache on all write operations 2022-03-23 19:14:33 +01:00
Yakov Olkhovskiy
1631656803
Merge pull request #34353 from zvonand/issue_33147
Extended usage of Milliseconds, Microseconds, Nanoseconds
2022-03-23 13:33:07 -04:00
kssenii
e39aba37a2 Minor changes 2022-03-23 18:11:52 +01:00