Commit Graph

70 Commits

Author SHA1 Message Date
alesapin
0f4056fd95 Add additional size check in debug mode 2020-12-09 14:23:37 +03:00
Nicolae Vartolomei
040aba9f85 Add uuid.txt to checksums for parts stored on disk
We are breaking backwards compatibility anyway (but agted by a setting)
2020-11-20 13:49:17 +00:00
Nicolae Vartolomei
425dc4b11b Add unique identifiers IMergeTreeDataPart structure
For now uuids are not generated at all, they are present only if the
part is updated manually (as you can see in the integration test).

The only place where they can be seen today by an end user is in
`system.parts` table. I was looking for hiding this column behind an
option but couldn't find an easy way to do that.

Likely this is also required for WAL, but need to think how not to break
compatibility.

Relates to #13574, https://github.com/ClickHouse/ClickHouse/issues/13574

Next 1: In the upcoming PR the plan is to integrate de-duplication based on
these fingerprints in the query pipeline.

Next 2: We'll enable automatic generation of uuids and come up with a
way for conditionally sending uuids when processing distributed queries
only when part movement is in progress.
2020-11-19 13:14:25 +00:00
Mikhail Filimonov
41971e073a
Fix typos reported by codespell 2020-10-27 12:04:03 +01:00
Alexey Milovidov
2619efadc8 Fix multiple issues with memory tracking 2020-10-21 03:31:34 +03:00
Vladimir Chebotarev
aa5f207fd4
Added disable_merges option for volumes in multi-disk configuration (#13956)
Co-authored-by: Alexander Kazakov <Akazz@users.noreply.github.com>
2020-10-20 18:10:24 +03:00
Pavel Kovalenko
ed61c5681b Use 'moving' directory instead of 'detached' when move part to another disk/volume. 2020-10-15 16:55:13 +03:00
Pavel Kovalenko
69c126f1f1 Possibility to move part to another disk/volume if first attempt was failed. 2020-10-07 14:35:28 +03:00
Pavel Kovalenko
69b4bc6f33 Proper error handling during insert into MergeTree with S3. 2020-10-06 12:38:00 +03:00
Anton Popov
3b9be70c8c Merge remote-tracking branch 'upstream/master' into HEAD 2020-09-05 18:53:55 +03:00
alesapin
acc0ee0657 Apply TTL if it's not calculated for part 2020-09-03 11:59:41 +03:00
Anton Popov
4fe57fc169 Merge remote-tracking branch 'upstream/master' into HEAD 2020-09-02 01:29:44 +03:00
alesapin
067eb4599d Fix empty columns case 2020-08-31 16:39:27 +03:00
alesapin
77faf9587f Better interface 2020-08-28 12:07:20 +03:00
alesapin
d426751e26 Default codec if part is not stored on disk 2020-08-27 16:32:23 +03:00
alesapin
1225c09a02 Fix default codec for in-memory parts 2020-08-27 11:35:55 +03:00
alesapin
44eb702fbe Remove redundant code 2020-08-26 22:41:57 +03:00
alesapin
3a680dc8fb Fix removal bug 2020-08-26 22:40:04 +03:00
alesapin
1fb93a8bf2 Remove read helpers 2020-08-26 18:57:06 +03:00
alesapin
18eb0dbcaa Better remove 2020-08-26 18:39:11 +03:00
alesapin
2fc80189af Add default compression codec to merge tree data part 2020-08-26 18:29:46 +03:00
Anton Popov
4df5c72878 Merge remote-tracking branch 'upstream/master' into HEAD 2020-08-24 12:57:27 +03:00
alexey-milovidov
7b53a0ef33 Revert "Added allow_merges option for volumes in multi-disk configuration (#13402)"
This reverts commit 1e2616542a.
2020-08-21 18:44:29 +03:00
Vladimir Chebotarev
1e2616542a
Added allow_merges option for volumes in multi-disk configuration (#13402) 2020-08-21 12:04:13 +03:00
alesapin
1707f84a44 Less strict check and rare rename bug 2020-07-27 12:42:37 +03:00
alesapin
78d357f0d2 Add a test for sticking mutations bug 2020-07-25 17:42:20 +03:00
alesapin
b3ec3b1ed1 Fix loading parts without checksums 2020-07-16 13:54:49 +03:00
Anton Popov
ca346ea13c rename fsync-related settings 2020-07-03 02:41:37 +03:00
Anton Popov
53e955c6dd several fixes 2020-06-29 23:36:18 +03:00
Anton Popov
b2aa565a37 allow to turn on fsync on inserts, merges and fetches 2020-06-27 17:52:16 +03:00
alesapin
6f1824f0ea Correct merge with master 2020-06-26 14:30:23 +03:00
alesapin
e9c47dc89c Merge branch 'master' into CurtizJ-polymorphic-parts 2020-06-26 14:27:19 +03:00
alesapin
dffdece350 getColumns in StorageInMemoryMetadta (only compilable) 2020-06-17 19:39:58 +03:00
alesapin
1afdebeebd Primary key in storage metadata 2020-06-17 15:39:20 +03:00
alesapin
ab61abccc1 Partition key in StorageInMemoryMetadata 2020-06-17 13:34:23 +03:00
Anton Popov
df3dfd5b81 fix clang-tidy build 2020-06-04 01:00:02 +03:00
Anton Popov
11c4e9dde3 in-memory parts: fix 'check' query 2020-06-03 22:19:49 +03:00
Anton Popov
1ce09e1faa Merge remote-tracking branch 'upstream/master' into polymorphic-parts 2020-06-03 16:27:54 +03:00
Anton Popov
caf1e4e8cc in-memory-parts: fixes 2020-06-03 12:51:23 +03:00
Alexey Milovidov
b78c48310b Whitespaces 2020-05-30 01:37:44 +03:00
Anton Popov
c919840722 in-memory parts: partition commands 2020-05-29 18:02:12 +03:00
Anton Popov
d8342e5b12 in-memory parts: better restore and clear stale wal files 2020-05-27 23:05:55 +03:00
alesapin
c7cda399c7 Merge branch 'master' into consistent_metadata2 2020-05-27 12:51:56 +03:00
alesapin
8614487a5a Fix primary key check 2020-05-26 16:46:19 +03:00
Alexey Milovidov
c858f4d89c Clarifications 2020-05-26 02:47:21 +03:00
alesapin
c3a6571036 Compilable code 2020-05-25 20:22:20 +03:00
alesapin
6281dd6893
Merge pull request #11115 from ClickHouse/consistent_metadata
Refactoring in storage metadata (more consistent keys)
2020-05-25 13:09:56 +03:00
Alexey Milovidov
7e1813825b Return old names of macros 2020-05-24 01:24:01 +03:00
Alexey Milovidov
e1695feb7f Apply all transformations again 2020-05-23 23:40:32 +03:00
Alexey Milovidov
85f84550ba Progress on task 2020-05-23 23:37:37 +03:00