thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-18 04:12:19 +00:00
Code Issues Packages Projects Releases Wiki Activity
68,214 Commits 9,324 Branches 1,783 Tags 2.2 GiB
3ba2d114d5
Commit Graph

528 Commits

Author SHA1 Message Date
Azat Khuzhin
de9689379f Catch ErrnoException during parts cleaning 
Avoids noisy messages like:

    2021.06.15 09:27:35.546982 [ 58176 ] {} <Error> auto DB::IBackgroundJobExecutor::jobExecutingTask()::(anonymous class)::operator()() const: Code: 481, e.displayText() = DB::ErrnoException: Cannot check modification time for file: /var/lib/clickhouse/store/c52/c52619f8-99bd-435e-8526-19f899bdb35e/tmp_insert_202106_7010_7010_0/, errno: 2, strerror: No such file or directory, Stack trace (when copying this message, always include the lines below):

    0. DB::Exception::Exception() @ 0x903209a in /usr/lib/debug/.build-id/be/ea088a1830cb2f.debug
    1. DB::throwFromErrnoWithPath() @ 0x9033089 in /usr/lib/debug/.build-id/be/ea088a1830cb2f.debug
    2. FS::getModificationTime() @ 0x907b166 in /usr/lib/debug/.build-id/be/ea088a1830cb2f.debug
    3. DB::DiskLocal::getLastModified() @ 0xf9a6669 in /usr/lib/debug/.build-id/be/ea088a1830cb2f.debug
    4. DB::MergeTreeData::clearOldTemporaryDirectories(long) @ 0x1069a6de in /usr/lib/debug/.build-id/be/ea088a1830cb2f.debug
 2021-06-16 06:56:55 +03:00
tavplubix
e2ecc51a1f
Merge pull request #25087 from ClickHouse/srmt_remove_copypaste 
Remove copypaste from StorageReplicatedMergeTree
 2021-06-15 12:43:55 +03:00
alexey-milovidov
30f0124e77
Merge branch 'master' into always-detach-parts-with-wrong-partition-id 2021-06-12 02:55:55 +03:00
Nikolai Kochetov
79c69ea316 Fix build. 2021-06-11 11:29:39 +03:00
Nikolai Kochetov
6222fcffe0 Never remove parts on start at all. Always move. 2021-06-10 15:47:55 +03:00
Nikolai Kochetov
aa90309abf Never remove parts on start at all. Always move. 2021-06-10 15:46:03 +03:00
Nikolai Kochetov
847dfb9914 Merge remote-tracking branch 'origin' into always-detach-parts-with-wrong-partition-id 2021-06-10 12:37:46 +03:00
Nikolai Kochetov
d52b87d47d Always detach proken parts with wrong partition id. 2021-06-10 12:37:33 +03:00
Amos Bird
4e358b28f1
Disable projection with final is used 2021-06-10 17:11:17 +08:00
Alexander Tokmakov
cef22688ff make code less ugly 2021-06-09 15:36:47 +03:00
Alexander Tokmakov
3ade38df82 remove copypaste 2021-06-08 22:17:45 +03:00
tavplubix
1aff716f18
Merge pull request #24777 from ClickHouse/fix_intersection_with_lost_part 
Do not crash on intersecting virtual parts
 2021-06-03 15:42:46 +03:00
Alexander Tokmakov
7d7c5638a5 Merge branch 'master' into fix_intersection_with_lost_part 2021-06-01 16:32:59 +03:00
Alexander Tokmakov
5969891611 do not crash on intersecting parts 2021-06-01 16:25:23 +03:00
Nikolai Kochetov
e615858b10 Merge branch 'master' into refactor-MergeTreeDataSelectExecutor 2021-06-01 16:25:04 +03:00
alexey-milovidov
c7bd3b8674
Merge pull request #23315 from CurtizJ/empty-parts-fix 
Fix waiting while dropping empty parts
 2021-06-01 13:52:49 +03:00
Nikolai Kochetov
b54ff696e5 Merge branch 'master' into refactor-MergeTreeDataSelectExecutor 2021-06-01 12:08:37 +03:00
tavplubix
e9ff0b6d70
Merge pull request #23657 from kssenii/poco-file-to-std-fs 
Poco::File to std::filesystem
 2021-05-31 23:17:02 +03:00
Nikolai Kochetov
b9aa404508 Review fixes. 2021-05-31 21:25:43 +03:00
Nikolai Kochetov
afc1fe7f3d Make ContextPtr const by default. 2021-05-31 17:49:02 +03:00
Alexander Tokmakov
16647fe8ce some unrelated fixes 2021-05-31 00:29:37 +03:00
kssenii
2a631aaf08 Final fixes 2021-05-29 00:34:44 +03:00
Anton Popov
336abfa475 Merge remote-tracking branch 'upstream/master' into HEAD 2021-05-28 01:58:30 +03:00
Nikolai Kochetov
936bd8a011 Merge branch 'master' into refactor-MergeTreeDataSelectExecutor 2021-05-27 20:47:39 +03:00
Nikolai Kochetov
1aeb705b20 Fix some tests. 2021-05-27 19:53:58 +03:00
Nikolai Kochetov
cbdf3752ef Part 3. 2021-05-27 16:40:33 +03:00
kssenii
73f16ee9ee Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-26 23:08:08 +03:00
alesapin
069d45790f
Merge pull request #24157 from kssenii/modulo-partition-key 
Adjust modulo function when used in partition key
 2021-05-26 10:30:54 +03:00
kssenii
31107816bb fs::absolute to fs::canonical 2021-05-24 20:37:54 +03:00
Anton Ivashkin
a98286b4c4 Merge master 2021-05-24 17:46:29 +03:00
kssenii
659e57e60f Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into modulo-partition-key 2021-05-23 22:46:00 +00:00
kssenii
bbf56e35ac Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-23 10:54:48 +03:00
kssenii
94b5763dbd Revert "Different approach to fix modulo" 
This reverts commit 665f1aa0f6.
 2021-05-21 16:14:01 +00:00
kssenii
30845a383f Fix comparisons with modulo key 2021-05-21 15:01:41 +00:00
kssenii
665f1aa0f6 Different approach to fix modulo 2021-05-21 14:30:08 +00:00
Anton Ivashkin
6b1b80f5cc Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into s3_zero_copy_replication 2021-05-21 16:01:48 +03:00
Anton Ivashkin
29336a4a34 Use keep_s3_on_delet flag instead of DeleteOnDestroyKeepS3 state 2021-05-21 15:29:10 +03:00
kssenii
3dee003f9b Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-20 19:20:09 +03:00
kssenii
477214a57f Fix partitionPrunner with modulo 2021-05-19 17:36:32 +00:00
mergify[bot]
230bec70fb
Merge branch 'master' into modulo-partition-key 2021-05-19 10:08:41 +00:00
alesapin
4d6c71801c
Merge branch 'master' into nv/parts-uuid-move-shard 2021-05-19 13:03:14 +03:00
tavplubix
8b13e73ac2
Merge pull request #24208 from ClickHouse/fix_intersecting_parts_again 
Addition to #23997
 2021-05-18 21:13:25 +03:00
Kseniia Sumarokova
0ab15d0bc4
Merge branch 'master' into modulo-partition-key 2021-05-18 17:45:32 +03:00
kssenii
5f083b779d A little better, fix checks 2021-05-18 14:25:19 +03:00
Alexander Tokmakov
20b9af7b26 do not allow concurrent alter partition 2021-05-18 00:20:58 +03:00
kssenii
9b8df78fdd Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-17 17:42:05 +03:00
Anton Popov
e27acc26be better dropPart and dropPartition 2021-05-17 17:26:36 +03:00
Anton Popov
e043cc45bf Merge remote-tracking branch 'upstream/master' into HEAD 2021-05-17 16:11:12 +03:00
Ilya Yatsishin
8ede5c736a
Merge pull request #23992 from MaxWk/enable-datatime64-version-column 
enable DateTime64 to be version column
 2021-05-17 14:34:15 +03:00
alesapin
17f229857c Merge branch 'master' into nvartolomei-parts-move 2021-05-17 13:52:48 +03:00
Anton Ivashkin
39a30b77fe Merge master 2021-05-17 11:47:48 +03:00
alesapin
eae2df6478
Merge pull request #23997 from ClickHouse/fix_intersecting_parts 
Try fix intersecting parts
 2021-05-17 10:46:58 +03:00
kssenii
3b1bf2bae6 Poco::Path substitution 2021-05-16 23:38:23 +03:00
kssenii
c7f51fe60e Modulo adjustment for partition key 2021-05-15 19:51:02 +00:00
Amos Bird
8de016d33a
Fix empty key projection query analysis 2021-05-14 22:26:26 +08:00
Alexander Tokmakov
41776ba5d9 fix 2021-05-14 15:55:30 +03:00
Alexander Tokmakov
970ad94851 Merge branch 'master' into fix_intersecting_parts 2021-05-14 13:17:02 +03:00
Alexander Tokmakov
2da12f0b59 fix intersection of drop range and merge 2021-05-13 17:04:36 +03:00
Alexander Tokmakov
e114c7eb8b fix virtual parts in REPLACE_RANGE 2021-05-13 14:29:59 +03:00
kssenii
0527f0ea33 Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-12 16:54:18 +03:00
kssenii
393b026f2e Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-11 16:49:23 +03:00
alesapin
46e136b5c4
Merge branch 'master' into nv/parts-uuid-move-shard 2021-05-11 15:36:40 +03:00
Amos Bird
074c2ead9c
Fix 2021-05-11 18:22:23 +08:00
Amos Bird
5e9fab6d10
Comment debug info out 2021-05-11 18:12:29 +08:00
Amos Bird
5836e2e277
Fix issues in normal projection 2021-05-11 18:12:29 +08:00
Amos Bird
ddd4256a15
More fixes 2021-05-11 18:12:28 +08:00
Amos Bird
718c284437
Fix more tests 2021-05-11 18:12:28 +08:00
Amos Bird
ba17acbd63
Fix tests 2021-05-11 18:12:28 +08:00
Amos Bird
a3fd0b6f2e
Fix tests 2021-05-11 18:12:28 +08:00
Amos Bird
42f161b1dc
Fix more 2021-05-11 18:12:28 +08:00
Amos Bird
9c069ebdbf
support prewhere, row_filter, read_in_order and decent projection selection 
TODO set index analysis in projection
 2021-05-11 18:12:27 +08:00
Amos Bird
35961c0c5d
Use ActionsDAG to rewrite projection queries 2021-05-11 18:12:27 +08:00
Nikolai Kochetov
29d7038e30
Add ProjectionDescription::Type enum. Remove some checks. 2021-05-11 18:12:27 +08:00
Amos Bird
ebaf42a448
Reformat and fix some tests 2021-05-11 18:12:27 +08:00
Nikolai Kochetov
672cfedd13
Disable normal projection by the number of granules. 2021-05-11 18:12:26 +08:00
Amos Bird
483bd165e7
Check if pipeline is simple and add more comments 2021-05-11 18:12:26 +08:00
Nikolai Kochetov
3296c9292f
Try to merge projectons faster. 2021-05-11 18:12:26 +08:00
Amos Bird
264cff6415
Projections 
TODO (suggested by Nikolai)

1. Build query plan fro current query (inside storage::read) up to WithMergableState
2. Check, that plan is simple enough: Aggregating - Expression - Filter - ReadFromStorage (or simplier)
3. Check, that filter is the same as filter in projection, and also expression calculates the same aggregation keys as in projection
4. Return WithMergableState if projection applies

3 will be easier to do with ActionsDAG, cause it sees all functions, and dependencies are direct (but it is possible with ExpressionActions also)

Also need to figure out how prewhere works for projections, and
row_filter_policies.

wip
 2021-05-11 18:12:23 +08:00
Kruglov Pavel
49e7ed6e60
Merge pull request #23673 from amosbird/partitionvalue 
Add _partition_value virtual column
 2021-05-11 11:23:00 +03:00
Alexander Tokmakov
eef899ae63 better code, add test 2021-05-10 21:03:37 +03:00
万康
308b964ca4 enable DateTime64 to be version column 2021-05-10 20:30:52 +08:00
kssenii
ab1a05a1f4 Poco::Path to fs::path, less concatination 2021-05-09 14:59:49 +03:00
Alexey Milovidov
ad88819ee4 Fix a bunch of warnings from PVS-Studio 2021-05-08 19:13:10 +03:00
Alexey Milovidov
e905883c75 More fixes for PVS-Studio 2021-05-08 19:12:31 +03:00
kssenii
02288359c5 Less manual concatenation of paths 2021-05-08 13:59:55 +03:00
kssenii
35f999bf04 Poco::createFile to fs::createFile 2021-05-08 02:41:47 +03:00
kssenii
140bf7e2ea Less Poco 2021-05-07 15:29:26 +03:00
kssenii
ee06936596 Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-01 17:24:31 +03:00
feng lv
aed2f337e9 Fix CLEAR COLUMN does not work after #21303 2021-04-30 05:02:32 +00:00
kssenii
deb4903af8 Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-04-28 20:57:13 +03:00
kssenii
eeb71672a0 Change in Storages/* 2021-04-27 16:49:37 +03:00
Nicolae Vartolomei
1fa5871ff7 Fix bad rebase and introduce part_moves_between_shards_enable setting 2021-04-27 14:20:13 +01:00
Nicolae Vartolomei
53d57ffb52 Part movement between shards 
Integrate query deduplication from #17348
 2021-04-27 14:20:12 +01:00
Amos Bird
8a3b5c1fab
Add _partition_value virtual column 2021-04-27 16:15:59 +08:00
mergify[bot]
a5da512e39
Merge branch 'master' into pid 2021-04-24 00:37:39 +00:00
alexey-milovidov
c66e715f49
Merge pull request #23433 from ClickHouse/tavplubix-patch-1 
Fix `Cannot unlink file` in `dropIfEmpty()` with multidisk confuguration
 2021-04-24 03:34:06 +03:00
alexey-milovidov
cd1f14ba61
Update MergeTreeData.cpp 2021-04-24 03:33:56 +03:00
hexiaoting
d24946f475 Fix bug when modify column without type 2021-04-22 14:48:57 +08:00
tavplubix
1a776b47a7
Update MergeTreeData.cpp 2021-04-21 15:34:42 +03:00
Amos Bird
50f2e488bd
Fix invalid virtual column expr 2021-04-21 10:29:03 +08:00