HuFuwang
4ac8860cea
refine code in MergeTreeData::loadDataParts to avoid parsing WAL file as data part.
2021-02-10 10:13:43 +08:00
tavplubix
86530e603c
Merge pull request #20027 from bharatnc/ncb/event_time_micro_part_log
...
add event_time_microseconds field to system.part_log table
2021-02-09 14:46:40 +03:00
Nicolae Vartolomei
a8f37e7704
Remove dead code
2021-02-08 17:19:35 +00:00
bharatnc
60c8dfa22c
PartLog - populate event_time_microseconds for MergeParts
2021-02-08 07:25:44 -08:00
Amos Bird
109a392e0c
Fix ubsan
2021-02-08 19:41:16 +08:00
Amos Bird
ab5c7b75a4
Delay or throw insertion when too many inactive parts
2021-02-07 19:03:55 +08:00
alesapin
011109c82a
Merge pull request #17348 from xjewer/alex/CLICKHOUSE-606_deduplication_UUID
...
CLICKHOUSE-606: query deduplication based on parts' UUID
2021-02-05 22:47:34 +03:00
Pavel Kovalenko
b7151aa754
Merge remote-tracking branch 'origin/master' into disk-s3-backup-restore-metadata
...
# Conflicts:
# src/Disks/DiskDecorator.h
# src/Disks/IDisk.h
# src/Disks/S3/DiskS3.cpp
2021-02-03 14:22:18 +03:00
Aleksei Semiglazov
921518db0a
CLICKHOUSE-606: query deduplication based on parts' UUID
...
* add the query data deduplication excluding duplicated parts in MergeTree family engines.
query deduplication is based on parts' UUID which should be enabled first with merge_tree setting
assign_part_uuids=1
allow_experimental_query_deduplication setting is to enable part deduplication, default ot false.
data part UUID is a mechanism of giving a data part a unique identifier.
Having UUID and deduplication mechanism provides a potential of moving parts
between shards preserving data consistency on a read path:
duplicated UUIDs will cause root executor to retry query against on of the replica explicitly
asking to exclude encountered duplicated fingerprints during a distributed query execution.
NOTE: this implementation don't provide any knobs to lock part and hence its UUID. Any mutations/merge will
update part's UUID.
* add _part_uuid virtual column, allowing to use UUIDs in predicates.
Signed-off-by: Aleksei Semiglazov <asemiglazov@cloudflare.com>
address comments
2021-02-02 16:53:39 +00:00
Nikolai Kochetov
19e4a33f9d
Merge pull request #19544 from amosbird/limitconcurrency
...
Per MergeTree table query limit
2021-02-01 16:09:12 +03:00
alesapin
5622e6daa6
Fix rare max_number_of_merges_with_ttl_in_pool limit overrun for non-replicated MergeTree
2021-01-27 14:56:12 +03:00
Amos Bird
66fe97d8bd
Per MergeTree table query limit
2021-01-26 14:03:31 +08:00
Pavel Kovalenko
1e3a059f64
Merge remote-tracking branch 'origin/master' into disk-s3-backup-restore-metadata
...
# Conflicts:
# src/Disks/DiskCacheWrapper.cpp
# src/Disks/S3/DiskS3.cpp
2021-01-18 13:39:49 +03:00
alexey-milovidov
5f189c5756
Merge pull request #19122 from ClickHouse/data-part-better-code
...
Add metrics for part number in MergeTree in ClickHouse
2021-01-16 00:20:15 +03:00
alexey-milovidov
971ff2ee0a
Merge pull request #19086 from ClickHouse/faster-parts-removal
...
Faster parts removal, more safe and efficient interface of IDisk
2021-01-15 20:37:35 +03:00
Alexey Milovidov
6a2a5e53ed
Slightly better code of IMergeTreeDataPart #18955
2021-01-15 15:15:13 +03:00
Alexey Milovidov
8276a1c8d2
Faster parts removal, more safe and efficient interface of IDisk
2021-01-14 19:24:13 +03:00
Anton Popov
ac426c3da6
restrict MODIFY TTL for tables created in old syntax
2021-01-14 15:32:20 +03:00
Pavel Kovalenko
b09862b7b9
Ability to backup-restore metadata files for DiskS3 (fixes and tests)
2021-01-12 20:18:40 +03:00
Pavel Kovalenko
0856b2c514
Ability to backup-restore metadata files for DiskS3 (fixes and tests)
2021-01-11 20:37:08 +03:00
alexey-milovidov
a08db94343
Revert "Add metrics for part number in MergeTree in ClickHouse"
2021-01-07 16:40:52 +03:00
weeds085490
5f5b86b485
Merge remote-tracking branch 'origin' into dev/add_metrics_for_parts
2021-01-06 17:32:45 +08:00
Nikita Mikhaylov
4165a58ce4
Merge pull request #11617 from nikitamikhaylov/parallel-parsing-input-format
...
Parallel formatting
2020-12-31 21:22:16 +04:00
alexey-milovidov
c2f0451e39
Merge pull request #18362 from ClickHouse/allow_low_cardinality_key_alter
...
Add ability to change some types for primary key
2020-12-31 15:27:52 +03:00
Nikita Mikhailov
60b4a36c4a
arcadia fix + live view fix + cleanup
2020-12-30 07:50:58 +03:00
Nikita Mikhailov
c0eef84c50
better
2020-12-29 04:15:26 +03:00
Nikita Mikhailov
d293ae5e8e
style
2020-12-28 20:18:53 +03:00
Nikita Mikhailov
dcfbe782c6
Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format
2020-12-23 05:20:22 +03:00
roverxu
e38ecee30e
fix style
2020-12-23 09:20:45 +08:00
徐炘
81f9623e2e
Merge branch 'master' into dev/add_metrics_for_parts
2020-12-22 19:04:28 +08:00
alesapin
35413635e7
Add ability to change some types for primary key
2020-12-22 13:41:12 +03:00
flynn
4e580f7b7c
Add asynchronous metrics on MergeTree tables data volume ( #17639 )
...
Co-authored-by: Alexander Kazakov <Akazz@users.noreply.github.com>
2020-12-22 13:34:35 +03:00
root
d6a69ee164
remove code
2020-12-22 18:12:28 +08:00
roverxu
f3fef24d30
fix deadlock
2020-12-21 16:30:27 +08:00
roverxu
8e72b6d31b
Merge remote-tracking branch 'origin' into dev/add_metrics_for_parts
2020-12-21 09:25:54 +08:00
Nikita Mikhailov
6a55e018e4
Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format
2020-12-18 02:25:32 +03:00
Alexey Milovidov
b25f126569
Added a test
2020-12-17 04:52:27 +03:00
Alexey Milovidov
9a017b77df
Merge branch 'master' into relax-too-many-parts-1
2020-12-17 04:40:39 +03:00
roverxu
1065ddef65
fix bug when drop data
2020-12-16 22:02:34 +08:00
Nikita Mikhaylov
5a47928431
save
2020-12-15 00:56:47 +03:00
Nikita Mikhaylov
f40f3ced2a
fix JSONEachRowArray
2020-12-15 00:56:46 +03:00
Nikita Mikhaylov
e0addac6fc
save changes
2020-12-15 00:56:46 +03:00
alesapin
0eec52b1fd
Merge pull request #17737 from ClickHouse/fix_segfault_in_distributed_out_stream
...
Fix segfault when 'not enough space'
2020-12-05 16:14:30 +03:00
detailyang
e7a151fd1e
hotfix:check in_memory_parts_enable_wal
2020-12-04 23:14:27 +08:00
roverxu
2a6c1463a1
Add metrics for part number in MergeTree
2020-12-04 11:42:58 +08:00
Alexander Tokmakov
bfbf150c67
fix segfault when 'not enough space'
2020-12-02 17:49:43 +03:00
Alexey Milovidov
2e6bedce91
Relax "Too many parts" threshold, part 1
2020-11-29 18:08:02 +03:00
feng lv
7e3524caa1
fix spelling errors
2020-11-28 08:17:20 +00:00
tavplubix
5cc9cb01cd
Merge pull request #16751 from amosbird/globalcontext
...
Make global_context consistent.
2020-11-22 18:46:17 +03:00
Amos Bird
1d9d586e20
Make global_context consistent.
2020-11-20 18:23:14 +08:00
Nicolae Vartolomei
746f8e45f5
All new parts must have uuids
2020-11-19 13:18:03 +00:00
Anton Popov
de5ead0c40
add setting remove_empty_parts
2020-11-15 05:24:47 +03:00
alexey-milovidov
915d96fee0
Merge pull request #16955 from nikitamikhaylov/fix-typos
...
Fix typos
2020-11-14 17:17:37 +03:00
Nikolai Kochetov
ed98b48845
Merge pull request #16618 from ClickHouse/actions-dag-f14
...
Actions dag 2
2020-11-13 18:34:51 +03:00
nikitamikhaylov
c0940f2ebd
done
2020-11-13 02:27:18 +03:00
Anton Popov
4cecd89e84
fix dropping of empty parts
2020-11-12 20:37:54 +03:00
Anton Popov
0aa35d53c7
Merge remote-tracking branch 'upstream/master' into HEAD
2020-11-12 14:31:58 +03:00
alesapin
2cac5e5d97
Merge pull request #16875 from CurtizJ/refactor-alter-partition
...
Avoid code duplication in alterPartition
2020-11-12 13:19:59 +03:00
Nikolai Kochetov
46f70dd0de
Merge branch 'master' into actions-dag-f14
2020-11-12 11:54:44 +03:00
tavplubix
058aa8f85e
Merge pull request #16824 from ClickHouse/replace_stringstreams_with_buffers
...
Replace std::*stringstreams with DB::*Buffers
2020-11-12 01:11:44 +03:00
Anton Popov
2497598a49
Remove empty parts after they were pruned by TTL
2020-11-11 19:19:43 +03:00
Anton Popov
6f0dc08ba7
avoid code duplication in alterPartition
2020-11-11 16:34:07 +03:00
Nikolai Kochetov
1846bb3cac
Merge branch 'master' into actions-dag-f14
2020-11-11 13:08:57 +03:00
Nikolai Kochetov
c6575c9032
Update ExpressionActions constructur
2020-11-10 19:27:55 +03:00
Nikolai Kochetov
1db8e77371
Add comments. Update ActionsDAG::Index
2020-11-10 17:54:59 +03:00
Alexander Tokmakov
5cdfcfb307
remove other stringstreams
2020-11-09 22:12:44 +03:00
Nikolai Kochetov
6717c7a0af
Merge branch 'master' into actions-dag-f14
2020-11-09 14:57:48 +03:00
alesapin
72046313d4
Merge branch 'master' into nvartolomei-drop-part
2020-11-09 10:02:02 +03:00
Alexey Milovidov
fd84d16387
Fix "server failed to start" error
2020-11-07 03:14:53 +03:00
Nikolai Kochetov
9aeb757da4
Merge branch 'master' into actions-dag-f14
2020-11-06 15:04:20 +03:00
alesapin
faca9d9b91
Check max bytes on disk
2020-11-03 18:41:46 +03:00
Nikolai Kochetov
07a7c46b89
Refactor ExpressionActions [Part 3]
2020-11-03 14:28:28 +03:00
alesapin
dac6705995
DROP PART and clear blocks in a single zookeeper op
2020-11-02 20:30:59 +03:00
alesapin
39e47c5338
Less strange lambdas
2020-10-23 11:54:00 +03:00
alesapin
7a63b373f9
Merge branch 'blind_fix_of_read_in_order' into no_background_pool_no_more
2020-10-22 15:47:11 +03:00
Vladimir Chebotarev
aa5f207fd4
Added disable_merges option for volumes in multi-disk configuration ( #13956 )
...
Co-authored-by: Alexander Kazakov <Akazz@users.noreply.github.com>
2020-10-20 18:10:24 +03:00
alesapin
7edbe4d276
Add some comments
2020-10-16 13:12:31 +03:00
alesapin
4014e0f08d
Something very similar to working code
2020-10-14 15:32:35 +03:00
alesapin
efd3126b5d
Moving pool
2020-10-14 10:22:48 +03:00
nikitamikhaylov
88a5302759
Merge branch 'master' of github.com:ClickHouse/ClickHouse into copier-backquotes
2020-10-09 16:26:31 +03:00
alesapin
8ec58c17f3
More consistent metadata usage
2020-10-05 19:41:46 +03:00
alesapin
8bc9ede711
Fix clang tidy
2020-10-01 09:53:47 +03:00
alesapin
97c2579fb9
Use separate function
2020-09-30 22:50:23 +03:00
alesapin
ac9f0ff4da
More strict check for version column
2020-09-30 22:44:35 +03:00
alesapin
65a05a970d
Better message on error
2020-09-29 15:07:01 +03:00
Nikita Mikhaylov
a4832ed5ce
Merge branch 'master' into HyaZz-master
2020-09-28 13:55:45 +03:00
alexey-milovidov
1fcebce926
Merge pull request #15000 from Jokser/disable-ttl-move-on-insert
...
Option to disable TTL move on data part insert
2020-09-19 13:45:02 +03:00
Pavel Kovalenko
6b4b5898a7
Renaming.
2020-09-18 18:41:14 +03:00
Pavel Kovalenko
0da19ab46d
Add option to disable ttl move on data part insert
2020-09-18 18:30:00 +03:00
alesapin
b07c00ef7c
Ugly, but probably working implementation
2020-09-17 18:33:50 +03:00
Alexey Milovidov
c1e9ce2cf2
Remove questionable code
2020-09-17 17:38:06 +03:00
alesapin
6c369c8dc6
Fix merge error
2020-09-17 15:01:03 +03:00
alesapin
40b2f203b6
Merge branch 'master' into manual-write-duplicate-parts-to-replicas
2020-09-17 13:21:00 +03:00
Anton Popov
af4089c606
Merge remote-tracking branch 'upstream/master' into HEAD
2020-09-11 00:39:21 +03:00
alesapin
485b104898
Use max ttl and add introspection to system parts about recompression
2020-09-09 12:15:42 +03:00
alesapin
f274ffc9d1
Add comments
2020-09-07 10:59:14 +03:00
alesapin
fecb2f1311
Less copypaste
2020-09-04 17:08:43 +03:00
alesapin
4079636f81
Merge
2020-09-04 13:52:51 +03:00
alesapin
82c56349a5
Some comments
2020-09-04 13:08:09 +03:00
alesapin
dcbddbb8d0
Add recompression ttls tests and fix bugs
2020-09-02 19:15:41 +03:00
