tavplubix
e9ff0b6d70
Merge pull request #23657 from kssenii/poco-file-to-std-fs
...
Poco::File to std::filesystem
2021-05-31 23:17:02 +03:00
Nikolai Kochetov
b9aa404508
Review fixes.
2021-05-31 21:25:43 +03:00
Alexander Tokmakov
5692db736c
Merge branch 'master' into mvcc_prototype
2021-05-31 21:11:22 +03:00
Nikolai Kochetov
afc1fe7f3d
Make ContextPtr const by default.
2021-05-31 17:49:02 +03:00
Alexander Tokmakov
16647fe8ce
some unrelated fixes
2021-05-31 00:29:37 +03:00
kssenii
2a631aaf08
Final fixes
2021-05-29 00:34:44 +03:00
Anton Popov
336abfa475
Merge remote-tracking branch 'upstream/master' into HEAD
2021-05-28 01:58:30 +03:00
Nikolai Kochetov
936bd8a011
Merge branch 'master' into refactor-MergeTreeDataSelectExecutor
2021-05-27 20:47:39 +03:00
Nikolai Kochetov
1aeb705b20
Fix some tests.
2021-05-27 19:53:58 +03:00
Nikolai Kochetov
cbdf3752ef
Part 3.
2021-05-27 16:40:33 +03:00
kssenii
73f16ee9ee
Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs
2021-05-26 23:08:08 +03:00
alesapin
069d45790f
Merge pull request #24157 from kssenii/modulo-partition-key
...
Adjust modulo function when used in partition key
2021-05-26 10:30:54 +03:00
kssenii
31107816bb
fs::absolute to fs::canonical
2021-05-24 20:37:54 +03:00
Anton Ivashkin
a98286b4c4
Merge master
2021-05-24 17:46:29 +03:00
kssenii
659e57e60f
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into modulo-partition-key
2021-05-23 22:46:00 +00:00
kssenii
bbf56e35ac
Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs
2021-05-23 10:54:48 +03:00
kssenii
94b5763dbd
Revert "Different approach to fix modulo"
...
This reverts commit 665f1aa0f6
2021-05-21 16:14:01 +00:00
kssenii
30845a383f
Fix comparisons with modulo key
2021-05-21 15:01:41 +00:00
kssenii
665f1aa0f6
Different approach to fix modulo
2021-05-21 14:30:08 +00:00
Anton Ivashkin
6b1b80f5cc
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into s3_zero_copy_replication
2021-05-21 16:01:48 +03:00
Anton Ivashkin
29336a4a34
Use keep_s3_on_delet flag instead of DeleteOnDestroyKeepS3 state
2021-05-21 15:29:10 +03:00
kssenii
3dee003f9b
Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs
2021-05-20 19:20:09 +03:00
kssenii
477214a57f
Fix partitionPrunner with modulo
2021-05-19 17:36:32 +00:00
mergify[bot]
230bec70fb
Merge branch 'master' into modulo-partition-key
2021-05-19 10:08:41 +00:00
alesapin
4d6c71801c
Merge branch 'master' into nv/parts-uuid-move-shard
2021-05-19 13:03:14 +03:00
tavplubix
8b13e73ac2
Merge pull request #24208 from ClickHouse/fix_intersecting_parts_again
...
Addition to #23997
2021-05-18 21:13:25 +03:00
Alexander Tokmakov
529d1aeb19
fix merges of uncommitted parts
2021-05-18 20:07:29 +03:00
Kseniia Sumarokova
0ab15d0bc4
Merge branch 'master' into modulo-partition-key
2021-05-18 17:45:32 +03:00
kssenii
5f083b779d
A little better, fix checks
2021-05-18 14:25:19 +03:00
Alexander Tokmakov
f2ab5a05c6
Merge branch 'master' into mvcc_prototype
2021-05-18 13:36:38 +03:00
Alexander Tokmakov
1fd6142c1a
try enable merges
2021-05-18 13:35:48 +03:00
Alexander Tokmakov
20b9af7b26
do not allow concurrent alter partition
2021-05-18 00:20:58 +03:00
kssenii
9b8df78fdd
Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs
2021-05-17 17:42:05 +03:00
Anton Popov
e27acc26be
better dropPart and dropPartition
2021-05-17 17:26:36 +03:00
Anton Popov
e043cc45bf
Merge remote-tracking branch 'upstream/master' into HEAD
2021-05-17 16:11:12 +03:00
Ilya Yatsishin
8ede5c736a
Merge pull request #23992 from MaxWk/enable-datatime64-version-column
...
enable DateTime64 to be version column
2021-05-17 14:34:15 +03:00
alesapin
17f229857c
Merge branch 'master' into nvartolomei-parts-move
2021-05-17 13:52:48 +03:00
Anton Ivashkin
39a30b77fe
Merge master
2021-05-17 11:47:48 +03:00
alesapin
eae2df6478
Merge pull request #23997 from ClickHouse/fix_intersecting_parts
...
Try fix intersecting parts
2021-05-17 10:46:58 +03:00
kssenii
3b1bf2bae6
Poco::Path substitution
2021-05-16 23:38:23 +03:00
kssenii
c7f51fe60e
Modulo adjustment for partition key
2021-05-15 19:51:02 +00:00
Amos Bird
8de016d33a
Fix empty key projection query analysis
2021-05-14 22:26:26 +08:00
Alexander Tokmakov
41776ba5d9
fix
2021-05-14 15:55:30 +03:00
Alexander Tokmakov
970ad94851
Merge branch 'master' into fix_intersecting_parts
2021-05-14 13:17:02 +03:00
Alexander Tokmakov
df2849be90
Merge branch 'master' into mvcc_prototype
2021-05-13 21:48:36 +03:00
Alexander Tokmakov
2da12f0b59
fix intersection of drop range and merge
2021-05-13 17:04:36 +03:00
Alexander Tokmakov
e114c7eb8b
fix virtual parts in REPLACE_RANGE
2021-05-13 14:29:59 +03:00
kssenii
0527f0ea33
Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs
2021-05-12 16:54:18 +03:00
kssenii
393b026f2e
Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs
2021-05-11 16:49:23 +03:00
alesapin
46e136b5c4
Merge branch 'master' into nv/parts-uuid-move-shard
2021-05-11 15:36:40 +03:00
Amos Bird
074c2ead9c
Fix
2021-05-11 18:22:23 +08:00
Amos Bird
5e9fab6d10
Comment debug info out
2021-05-11 18:12:29 +08:00
Amos Bird
5836e2e277
Fix issues in normal projection
2021-05-11 18:12:29 +08:00
Amos Bird
ddd4256a15
More fixes
2021-05-11 18:12:28 +08:00
Amos Bird
718c284437
Fix more tests
2021-05-11 18:12:28 +08:00
Amos Bird
ba17acbd63
Fix tests
2021-05-11 18:12:28 +08:00
Amos Bird
a3fd0b6f2e
Fix tests
2021-05-11 18:12:28 +08:00
Amos Bird
42f161b1dc
Fix more
2021-05-11 18:12:28 +08:00
Amos Bird
9c069ebdbf
support prewhere, row_filter, read_in_order and decent projection selection
...
TODO set index analysis in projection
2021-05-11 18:12:27 +08:00
Amos Bird
35961c0c5d
Use ActionsDAG to rewrite projection queries
2021-05-11 18:12:27 +08:00
Nikolai Kochetov
29d7038e30
Add ProjectionDescription::Type enum. Remove some checks.
2021-05-11 18:12:27 +08:00
Amos Bird
ebaf42a448
Reformat and fix some tests
2021-05-11 18:12:27 +08:00
Nikolai Kochetov
672cfedd13
Disable normal projection by the number of granules.
2021-05-11 18:12:26 +08:00
Amos Bird
483bd165e7
Check if pipeline is simple and add more comments
2021-05-11 18:12:26 +08:00
Nikolai Kochetov
3296c9292f
Try to merge projectons faster.
2021-05-11 18:12:26 +08:00
Amos Bird
264cff6415
Projections
...
TODO (suggested by Nikolai)
1. Build query plan fro current query (inside storage::read) up to WithMergableState
2. Check, that plan is simple enough: Aggregating - Expression - Filter - ReadFromStorage (or simplier)
3. Check, that filter is the same as filter in projection, and also expression calculates the same aggregation keys as in projection
4. Return WithMergableState if projection applies
3 will be easier to do with ActionsDAG, cause it sees all functions, and dependencies are direct (but it is possible with ExpressionActions also)
Also need to figure out how prewhere works for projections, and
row_filter_policies.
wip
2021-05-11 18:12:23 +08:00
Kruglov Pavel
49e7ed6e60
Merge pull request #23673 from amosbird/partitionvalue
...
Add _partition_value virtual column
2021-05-11 11:23:00 +03:00
Alexander Tokmakov
eef899ae63
better code, add test
2021-05-10 21:03:37 +03:00
万康
308b964ca4
enable DateTime64 to be version column
2021-05-10 20:30:52 +08:00
kssenii
ab1a05a1f4
Poco::Path to fs::path, less concatination
2021-05-09 14:59:49 +03:00
Alexey Milovidov
ad88819ee4
Fix a bunch of warnings from PVS-Studio
2021-05-08 19:13:10 +03:00
Alexey Milovidov
e905883c75
More fixes for PVS-Studio
2021-05-08 19:12:31 +03:00
kssenii
02288359c5
Less manual concatenation of paths
2021-05-08 13:59:55 +03:00
kssenii
35f999bf04
Poco::createFile to fs::createFile
2021-05-08 02:41:47 +03:00
kssenii
140bf7e2ea
Less Poco
2021-05-07 15:29:26 +03:00
kssenii
ee06936596
Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs
2021-05-01 17:24:31 +03:00
feng lv
aed2f337e9
Fix CLEAR COLUMN does not work after #21303
2021-04-30 05:02:32 +00:00
kssenii
deb4903af8
Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs
2021-04-28 20:57:13 +03:00
kssenii
eeb71672a0
Change in Storages/*
2021-04-27 16:49:37 +03:00
Nicolae Vartolomei
1fa5871ff7
Fix bad rebase and introduce part_moves_between_shards_enable setting
2021-04-27 14:20:13 +01:00
Nicolae Vartolomei
53d57ffb52
Part movement between shards
...
Integrate query deduplication from #17348
2021-04-27 14:20:12 +01:00
Amos Bird
8a3b5c1fab
Add _partition_value virtual column
2021-04-27 16:15:59 +08:00
mergify[bot]
a5da512e39
Merge branch 'master' into pid
2021-04-24 00:37:39 +00:00
alexey-milovidov
c66e715f49
Merge pull request #23433 from ClickHouse/tavplubix-patch-1
...
Fix `Cannot unlink file` in `dropIfEmpty()` with multidisk confuguration
2021-04-24 03:34:06 +03:00
alexey-milovidov
cd1f14ba61
Update MergeTreeData.cpp
2021-04-24 03:33:56 +03:00
hexiaoting
d24946f475
Fix bug when modify column without type
2021-04-22 14:48:57 +08:00
tavplubix
1a776b47a7
Update MergeTreeData.cpp
2021-04-21 15:34:42 +03:00
Amos Bird
50f2e488bd
Fix invalid virtual column expr
2021-04-21 10:29:03 +08:00
Anton Popov
42a0416a2b
better drop of empty parts
2021-04-20 05:31:08 +03:00
Anton Popov
66913c5865
fix waiting while dropping empty parts
2021-04-19 19:24:50 +03:00
Ivan Lezhankin
7a88ca31cc
Merge branch 'master' into async-insert
2021-04-19 18:33:43 +03:00
tavplubix
3d1885f08c
Merge pull request #23177 from marsno1/fix-exception-log
...
Fix exception message for "parts_to_throw_insert"
2021-04-16 18:15:07 +03:00
Anton Ivashkin
09379b2b8a
Fix Zero-Copy replication with several S3 volumes (issue 22679)
2021-04-16 12:34:48 +03:00
madianjun
9238d8e54a
Fix exception message for parts_to_throw_insert
2021-04-16 14:28:52 +08:00
Ivan Lezhankin
9e5d274467
Merge remote-tracking branch 'upstream/master' into async-insert
2021-04-14 20:16:05 +03:00
songenjie
564136ec46
[clickhouse][server][dll][alter]support fetch part
2021-04-13 12:40:33 +08:00
Denny Crane
eaa092cae2
Update MergeTreeData.cpp
...
Better error message.
2021-04-12 10:58:06 -03:00
Egor O'Sten
c8b572ed59
Readability of error message for ORDER BY
2021-04-11 20:42:05 -04:00
Egor O'Sten
c77dd36ca9
Readability of error message for ORDER BY
2021-04-11 20:40:00 -04:00
Azat Khuzhin
940ff6cb76
Do not use c-style cast for std::min(max_part_removal_threads)
2021-04-11 12:22:39 +03:00
Azat Khuzhin
6e71757814
Fix thread accounting for max_part_removal_threads
2021-04-11 11:43:30 +03:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr ( #22297 )
...
* Replace all Context references with std::weak_ptr
* Fix shared context captured by value
* Fix build
* Fix Context with named sessions
* Fix copy context
* Fix gcc build
* Merge with master and fix build
* Fix gcc-9 build
2021-04-11 02:33:54 +03:00
Azat Khuzhin
a6703c5ac2
Fix inactive_parts_to_throw_insert=0 with inactive_parts_to_delay_insert>0
2021-04-10 09:57:08 +03:00
Alexander Tokmakov
3422bd1742
check parts visibility for select
2021-04-08 20:20:45 +03:00
Alexander Tokmakov
f596906f47
add transaction counters
2021-04-05 14:41:34 +03:00
alesapin
8d5a787f6b
Merge branch 'master' into merge_tree_deduplication
2021-04-05 10:40:03 +03:00
Alexey Milovidov
54caa61af5
Fix comments
2021-04-04 12:33:06 +03:00
alesapin
0204f5dd35
Merge branch 'master' into merge_tree_deduplication
2021-04-03 15:24:26 +03:00
alesapin
b884593186
Followup fix
2021-04-02 20:55:05 +03:00
alesapin
ca30c91868
More comments
2021-04-02 20:54:24 +03:00
alesapin
fc3afe1087
Fix typos
2021-04-02 20:51:57 +03:00
alesapin
b105153d2c
Useful comment
2021-04-02 20:48:58 +03:00
alesapin
759dd79c76
Add tests
2021-04-02 19:45:18 +03:00
Mike Kot
c947280dfc
Merge remote-tracking branch 'upstream/master' into feature/attach-partition-local
2021-04-01 21:38:51 +03:00
Alexey Milovidov
230b2930b7
Merge branch 'master' into virtualutil
2021-03-30 00:43:00 +03:00
Ivan Lezhankin
de05803177
Merge branch 'master' into async-insert
2021-03-29 20:59:02 +03:00
Alexey Milovidov
2a8ac01cdb
Rename as suggested by Kita
2021-03-28 22:24:28 +03:00
Alexey Milovidov
6f9f7d118d
Fixes
2021-03-28 04:27:26 +03:00
Alexey Milovidov
50f712e198
Integrate mmap cache to the infrastructure
2021-03-28 04:10:30 +03:00
alexey-milovidov
f895bc895c
Merge pull request #22011 from ClickHouse/min_max_time_system_parts_datetime64
...
Expose DateTime64 minmax part index in system.parts and system.parts_columns
2021-03-25 16:02:33 +03:00
Mike Kot
285af08949
Merge remote-tracking branch 'upstream/master' into feature/attach-partition-local
2021-03-24 22:34:20 +03:00
alesapin
36898bdc4a
Merge pull request #21142 from Jokser/unfreeze-partitions
...
Add ALTER TABLE UNFREEZE command.
2021-03-23 14:05:38 +03:00
Alexey Milovidov
8d0210b510
Expose DateTime64 minmax part index in system.parts and system.parts_columns #18244
2021-03-23 01:16:41 +03:00
Alexey Milovidov
ce67afd9e3
Fix bad code #18244
2021-03-23 00:49:18 +03:00
Ivan Lezhankin
4f1fad97ab
Merge remote-tracking branch 'upstream/master' into async-insert
2021-03-19 23:41:01 +03:00
Nikolai Kochetov
c3c393a7aa
Merge branch 'master' into refactor-actions-dag
2021-03-18 14:33:07 +03:00
Mike Kot
831b90f272
Replaced the iteration algorithm in part finder
...
Now, instead of iterating through the directories, we iterate though
directories of on of the table disks (which doesn't give us a
substantial boost but is a bit neater to read).
- Updated the system.replication_queue command types.
- Fixed the part ptr being empty (added the checksum loading and
initialization).
- Removed extra logging.
2021-03-17 16:55:26 +03:00
Mike Kot
2ccdb7ef5c
Multiple small code and test updates
...
- Updated the docs to make everything clear.
- Multiple small logger fixes.
- Changed the attach_part command -- now it's after check for the
covering parts -- motivation is to do less work with the checksums
fetching.
- Better logging in the integration test.
2021-03-17 16:52:35 +03:00
Pavel Kovalenko
5627ae9067
Make MergeTreeData::getPartitionMatcher const
2021-03-17 12:12:55 +03:00
Pavel Kovalenko
a43c85b105
Add test for freeze/unfreeze partition with old MergeTree syntax
2021-03-16 16:36:05 +03:00
Pavel Kovalenko
39f3b17041
Merge remote-tracking branch 'origin/master' into unfreeze-partitions
2021-03-16 10:56:24 +03:00
alexey-milovidov
378e10f929
Merge pull request #21351 from fuwhu/refine-load-data-parts
...
Refine MergeTreeData::loadDataParts to not parse format version file and detached directory
2021-03-15 22:38:30 +03:00
Mike Kot
406d037ebb
Merge remote-tracking branch 'upstream/master' into feature/attach-partition-local
2021-03-15 18:41:47 +03:00
fuwhu
69f516cd47
Refine MergeTreeData::loadDataParts to not parse format version file and detached directory
2021-03-10 10:10:25 +08:00
Amos Bird
e52b2fb8ae
update
2021-03-10 02:56:46 +08:00
Amos Bird
2ec20c5d23
update and add tests
2021-03-08 17:38:07 +08:00
Amos Bird
2b11a2a855
Fix build
2021-03-08 11:10:35 +08:00
Amos Bird
e6522e1ebe
JBOD data balancer
2021-03-08 11:10:35 +08:00
Nikolai Kochetov
a669f7d641
Merge branch 'master' into refactor-actions-dag
2021-03-05 18:21:14 +03:00
Nikolai Kochetov
9a39459888
Refactor ActionsDAG
2021-03-04 20:38:12 +03:00
Amos Bird
909cb3c243
Fix again....
2021-03-04 22:27:07 +08:00
Amos Bird
9205fad8c7
Better
2021-03-04 19:43:03 +08:00
Amos Bird
93b661ad5a
partition id pruning
2021-03-04 19:43:03 +08:00
Ivan Lezhankin
74214865eb
Merge remote-tracking branch 'upstream/master' into async-insert
2021-03-04 14:14:28 +03:00
Ivan Lezhankin
b726845be2
[WIP]
2021-03-04 14:10:21 +03:00
Pavel Kovalenko
c4523ec74c
Add 'const auto' for variable.
2021-03-03 10:49:23 +03:00
alesapin
6b1005aea5
Merge pull request #21334 from ClickHouse/fix_alter_partition_key
...
Fix alter modify query for partition key and other metadata fields
2021-03-03 10:15:11 +03:00
Anton Popov
a4c00ab5dc
Merge pull request #21303 from ucasFL/forbid
...
Forbid to drop a column if it's referenced by materialized view
2021-03-03 02:55:06 +03:00
Pavel Kovalenko
483e63ca45
ALTER TABLE UNFREEZE rework.
2021-03-03 00:10:09 +03:00
Pavel Kovalenko
a3c0c35dff
ALTER TABLE UNFREEZE rework.
2021-03-02 23:28:42 +03:00
Nikolai Kochetov
4775ea305e
Remove index by name from ActionsDAG
2021-03-02 20:51:54 +03:00
alesapin
c29d7c7f49
Shutup clang tidy
2021-03-02 19:13:36 +03:00
alesapin
9ebf1b4fad
Get rid of separate minmax index fields
2021-03-02 13:33:54 +03:00
feng lv
a26c9e64a9
fix
...
fix
2021-03-02 03:20:03 +00:00
Mike Kot
4e1ac185b5
Merge remote-tracking branch 'upstream/master' into feature/attach-partition-local
2021-03-01 20:58:07 +03:00
alesapin
9c8afbeb53
Fix alter modify query for partition key and other metadata fields
2021-03-01 12:59:19 +03:00
Azat Khuzhin
8b58dba094
Drop unused HexWriteBuffer
2021-02-28 10:52:09 +03:00
feng lv
51021c1164
forbid to drop a column if it's referenced by materialized view
2021-02-28 05:24:39 +00:00
Pavel Kovalenko
ed44845f2c
Fixed backup_path for unfreeze
2021-02-25 22:21:22 +03:00
alesapin
5c6c318737
Restrict mutations for engines which doesn't support them
2021-02-25 13:07:48 +03:00
Pavel Kovalenko
0df783e0ef
Fix code style
2021-02-24 18:38:40 +03:00
Pavel Kovalenko
078dfce038
Add ALTER TABLE UNFREEZE command.
2021-02-24 17:28:30 +03:00
Anton Popov
48e188681c
do not start mutation for alters with wrong type conversion
2021-02-20 19:11:08 +03:00
Mike Kot
73f4740be5
Trying to re-write the solution by adding new command type
...
ATTACH_PART into the replicated log.
The LogEntry now also has the pre-calculated part checksum for this
entry type, which is later used while searching in the detached/ folder
2021-02-15 18:06:48 +03:00
alesapin
a2bb586170
Merge pull request #20335 from ClickHouse/fix_scheduling_of_non_successful_background_tasks
...
Fix backoff for failed background tasks in replicated merge tree
2021-02-12 10:06:04 +03:00
alexey-milovidov
7b55eb8c58
Merge pull request #20178 from amosbird/delayinactive
...
Delay or throw insertion when too many inactive parts
2021-02-11 20:52:04 +03:00
alesapin
3253638969
Fix backoff for failed background tasks in replicated merge tree
2021-02-11 14:46:18 +03:00
alesapin
56453256b7
Merge pull request #20300 from ClickHouse/restrict_to_drop_version_column
...
Don't allow to drop or rename version column
2021-02-11 11:04:00 +03:00
Anton Popov
365e2fb746
Merge pull request #20231 from fuwhu/refine-loadDataParts-code
...
refine code in MergeTreeData::loadDataParts to avoid parsing WAL file as data part
2021-02-10 22:42:31 +03:00
alesapin
891fce3275
Don't allow to drop or rename version column
2021-02-10 22:23:48 +03:00
Amos Bird
afb5846a24
refactor
2021-02-10 11:22:24 +08:00
HuFuwang
4ac8860cea
refine code in MergeTreeData::loadDataParts to avoid parsing WAL file as data part.
2021-02-10 10:13:43 +08:00
tavplubix
86530e603c
Merge pull request #20027 from bharatnc/ncb/event_time_micro_part_log
...
add event_time_microseconds field to system.part_log table
2021-02-09 14:46:40 +03:00
Nicolae Vartolomei
a8f37e7704
Remove dead code
2021-02-08 17:19:35 +00:00
bharatnc
60c8dfa22c
PartLog - populate event_time_microseconds for MergeParts
2021-02-08 07:25:44 -08:00
Amos Bird
109a392e0c
Fix ubsan
2021-02-08 19:41:16 +08:00
Amos Bird
ab5c7b75a4
Delay or throw insertion when too many inactive parts
2021-02-07 19:03:55 +08:00
alesapin
011109c82a
Merge pull request #17348 from xjewer/alex/CLICKHOUSE-606_deduplication_UUID
...
CLICKHOUSE-606: query deduplication based on parts' UUID
2021-02-05 22:47:34 +03:00
Pavel Kovalenko
b7151aa754
Merge remote-tracking branch 'origin/master' into disk-s3-backup-restore-metadata
...
# Conflicts:
# src/Disks/DiskDecorator.h
# src/Disks/IDisk.h
# src/Disks/S3/DiskS3.cpp
2021-02-03 14:22:18 +03:00
Aleksei Semiglazov
921518db0a
CLICKHOUSE-606: query deduplication based on parts' UUID
...
* add the query data deduplication excluding duplicated parts in MergeTree family engines.
query deduplication is based on parts' UUID which should be enabled first with merge_tree setting
assign_part_uuids=1
allow_experimental_query_deduplication setting is to enable part deduplication, default ot false.
data part UUID is a mechanism of giving a data part a unique identifier.
Having UUID and deduplication mechanism provides a potential of moving parts
between shards preserving data consistency on a read path:
duplicated UUIDs will cause root executor to retry query against on of the replica explicitly
asking to exclude encountered duplicated fingerprints during a distributed query execution.
NOTE: this implementation don't provide any knobs to lock part and hence its UUID. Any mutations/merge will
update part's UUID.
* add _part_uuid virtual column, allowing to use UUIDs in predicates.
Signed-off-by: Aleksei Semiglazov <asemiglazov@cloudflare.com>
address comments
2021-02-02 16:53:39 +00:00
Nikolai Kochetov
19e4a33f9d
Merge pull request #19544 from amosbird/limitconcurrency
...
Per MergeTree table query limit
2021-02-01 16:09:12 +03:00
alesapin
5622e6daa6
Fix rare max_number_of_merges_with_ttl_in_pool limit overrun for non-replicated MergeTree
2021-01-27 14:56:12 +03:00
Amos Bird
66fe97d8bd
Per MergeTree table query limit
2021-01-26 14:03:31 +08:00
Pavel Kovalenko
1e3a059f64
Merge remote-tracking branch 'origin/master' into disk-s3-backup-restore-metadata
...
# Conflicts:
# src/Disks/DiskCacheWrapper.cpp
# src/Disks/S3/DiskS3.cpp
2021-01-18 13:39:49 +03:00
alexey-milovidov
5f189c5756
Merge pull request #19122 from ClickHouse/data-part-better-code
...
Add metrics for part number in MergeTree in ClickHouse
2021-01-16 00:20:15 +03:00
alexey-milovidov
971ff2ee0a
Merge pull request #19086 from ClickHouse/faster-parts-removal
...
Faster parts removal, more safe and efficient interface of IDisk
2021-01-15 20:37:35 +03:00
Alexey Milovidov
6a2a5e53ed
Slightly better code of IMergeTreeDataPart #18955
2021-01-15 15:15:13 +03:00
Alexey Milovidov
8276a1c8d2
Faster parts removal, more safe and efficient interface of IDisk
2021-01-14 19:24:13 +03:00
Anton Popov
ac426c3da6
restrict MODIFY TTL for tables created in old syntax
2021-01-14 15:32:20 +03:00
Pavel Kovalenko
b09862b7b9
Ability to backup-restore metadata files for DiskS3 (fixes and tests)
2021-01-12 20:18:40 +03:00
Pavel Kovalenko
0856b2c514
Ability to backup-restore metadata files for DiskS3 (fixes and tests)
2021-01-11 20:37:08 +03:00
alexey-milovidov
a08db94343
Revert "Add metrics for part number in MergeTree in ClickHouse"
2021-01-07 16:40:52 +03:00
weeds085490
5f5b86b485
Merge remote-tracking branch 'origin' into dev/add_metrics_for_parts
2021-01-06 17:32:45 +08:00
Nikita Mikhaylov
4165a58ce4
Merge pull request #11617 from nikitamikhaylov/parallel-parsing-input-format
...
Parallel formatting
2020-12-31 21:22:16 +04:00
alexey-milovidov
c2f0451e39
Merge pull request #18362 from ClickHouse/allow_low_cardinality_key_alter
...
Add ability to change some types for primary key
2020-12-31 15:27:52 +03:00
Nikita Mikhailov
60b4a36c4a
arcadia fix + live view fix + cleanup
2020-12-30 07:50:58 +03:00
Nikita Mikhailov
c0eef84c50
better
2020-12-29 04:15:26 +03:00
Nikita Mikhailov
d293ae5e8e
style
2020-12-28 20:18:53 +03:00
Nikita Mikhailov
dcfbe782c6
Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format
2020-12-23 05:20:22 +03:00
roverxu
e38ecee30e
fix style
2020-12-23 09:20:45 +08:00
徐炘
81f9623e2e
Merge branch 'master' into dev/add_metrics_for_parts
2020-12-22 19:04:28 +08:00
alesapin
35413635e7
Add ability to change some types for primary key
2020-12-22 13:41:12 +03:00
flynn
4e580f7b7c
Add asynchronous metrics on MergeTree tables data volume ( #17639 )
...
Co-authored-by: Alexander Kazakov <Akazz@users.noreply.github.com>
2020-12-22 13:34:35 +03:00
root
d6a69ee164
remove code
2020-12-22 18:12:28 +08:00
roverxu
f3fef24d30
fix deadlock
2020-12-21 16:30:27 +08:00
roverxu
8e72b6d31b
Merge remote-tracking branch 'origin' into dev/add_metrics_for_parts
2020-12-21 09:25:54 +08:00
Nikita Mikhailov
6a55e018e4
Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format
2020-12-18 02:25:32 +03:00
Alexey Milovidov
b25f126569
Added a test
2020-12-17 04:52:27 +03:00
Alexey Milovidov
9a017b77df
Merge branch 'master' into relax-too-many-parts-1
2020-12-17 04:40:39 +03:00
roverxu
1065ddef65
fix bug when drop data
2020-12-16 22:02:34 +08:00
Nikita Mikhaylov
5a47928431
save
2020-12-15 00:56:47 +03:00
Nikita Mikhaylov
f40f3ced2a
fix JSONEachRowArray
2020-12-15 00:56:46 +03:00
Nikita Mikhaylov
e0addac6fc
save changes
2020-12-15 00:56:46 +03:00
alesapin
0eec52b1fd
Merge pull request #17737 from ClickHouse/fix_segfault_in_distributed_out_stream
...
Fix segfault when 'not enough space'
2020-12-05 16:14:30 +03:00
detailyang
e7a151fd1e
hotfix:check in_memory_parts_enable_wal
2020-12-04 23:14:27 +08:00
roverxu
2a6c1463a1
Add metrics for part number in MergeTree
2020-12-04 11:42:58 +08:00
Alexander Tokmakov
bfbf150c67
fix segfault when 'not enough space'
2020-12-02 17:49:43 +03:00
Alexey Milovidov
2e6bedce91
Relax "Too many parts" threshold, part 1
2020-11-29 18:08:02 +03:00
feng lv
7e3524caa1
fix spelling errors
2020-11-28 08:17:20 +00:00
tavplubix
5cc9cb01cd
Merge pull request #16751 from amosbird/globalcontext
...
Make global_context consistent.
2020-11-22 18:46:17 +03:00
Amos Bird
1d9d586e20
Make global_context consistent.
2020-11-20 18:23:14 +08:00
Nicolae Vartolomei
746f8e45f5
All new parts must have uuids
2020-11-19 13:18:03 +00:00
Anton Popov
de5ead0c40
add setting remove_empty_parts
2020-11-15 05:24:47 +03:00
alexey-milovidov
915d96fee0
Merge pull request #16955 from nikitamikhaylov/fix-typos
...
Fix typos
2020-11-14 17:17:37 +03:00
Nikolai Kochetov
ed98b48845
Merge pull request #16618 from ClickHouse/actions-dag-f14
...
Actions dag 2
2020-11-13 18:34:51 +03:00
nikitamikhaylov
c0940f2ebd
done
2020-11-13 02:27:18 +03:00
Anton Popov
4cecd89e84
fix dropping of empty parts
2020-11-12 20:37:54 +03:00
Anton Popov
0aa35d53c7
Merge remote-tracking branch 'upstream/master' into HEAD
2020-11-12 14:31:58 +03:00
alesapin
2cac5e5d97
Merge pull request #16875 from CurtizJ/refactor-alter-partition
...
Avoid code duplication in alterPartition
2020-11-12 13:19:59 +03:00
Nikolai Kochetov
46f70dd0de
Merge branch 'master' into actions-dag-f14
2020-11-12 11:54:44 +03:00
tavplubix
058aa8f85e
Merge pull request #16824 from ClickHouse/replace_stringstreams_with_buffers
...
Replace std::*stringstreams with DB::*Buffers
2020-11-12 01:11:44 +03:00
Anton Popov
2497598a49
Remove empty parts after they were pruned by TTL
2020-11-11 19:19:43 +03:00
Anton Popov
6f0dc08ba7
avoid code duplication in alterPartition
2020-11-11 16:34:07 +03:00
Nikolai Kochetov
1846bb3cac
Merge branch 'master' into actions-dag-f14
2020-11-11 13:08:57 +03:00
Nikolai Kochetov
c6575c9032
Update ExpressionActions constructur
2020-11-10 19:27:55 +03:00
Nikolai Kochetov
1db8e77371
Add comments. Update ActionsDAG::Index
2020-11-10 17:54:59 +03:00
Alexander Tokmakov
5cdfcfb307
remove other stringstreams
2020-11-09 22:12:44 +03:00
Nikolai Kochetov
6717c7a0af
Merge branch 'master' into actions-dag-f14
2020-11-09 14:57:48 +03:00
alesapin
72046313d4
Merge branch 'master' into nvartolomei-drop-part
2020-11-09 10:02:02 +03:00
Alexey Milovidov
fd84d16387
Fix "server failed to start" error
2020-11-07 03:14:53 +03:00
Nikolai Kochetov
9aeb757da4
Merge branch 'master' into actions-dag-f14
2020-11-06 15:04:20 +03:00
alesapin
faca9d9b91
Check max bytes on disk
2020-11-03 18:41:46 +03:00
Nikolai Kochetov
07a7c46b89
Refactor ExpressionActions [Part 3]
2020-11-03 14:28:28 +03:00
alesapin
dac6705995
DROP PART and clear blocks in a single zookeeper op
2020-11-02 20:30:59 +03:00
alesapin
39e47c5338
Less strange lambdas
2020-10-23 11:54:00 +03:00
alesapin
7a63b373f9
Merge branch 'blind_fix_of_read_in_order' into no_background_pool_no_more
2020-10-22 15:47:11 +03:00
Vladimir Chebotarev
aa5f207fd4
Added disable_merges option for volumes in multi-disk configuration ( #13956 )
...
Co-authored-by: Alexander Kazakov <Akazz@users.noreply.github.com>
2020-10-20 18:10:24 +03:00
alesapin
7edbe4d276
Add some comments
2020-10-16 13:12:31 +03:00
alesapin
4014e0f08d
Something very similar to working code
2020-10-14 15:32:35 +03:00
alesapin
efd3126b5d
Moving pool
2020-10-14 10:22:48 +03:00
