thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 03:42:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
85,959 Commits 9,315 Branches 1,783 Tags 2.1 GiB
e7e6125b46
Commit Graph

290 Commits

Author SHA1 Message Date
alesapin
c15d7e009d Some initial code 2021-03-31 18:20:30 +03:00
Anton Popov
372a1b1fe7 Merge remote-tracking branch 'upstream/master' into HEAD 2021-03-29 19:57:49 +03:00
Anton Popov
577d571300 ColumnSparse: initial implementation 2021-03-29 19:54:24 +03:00
Anton Popov
ea82e7725f
Merge pull request #21562 from CurtizJ/serialization-refactoring-4 
Refactoring of data types serialization
 2021-03-29 16:36:44 +03:00
alexey-milovidov
f895bc895c
Merge pull request #22011 from ClickHouse/min_max_time_system_parts_datetime64 
Expose DateTime64 minmax part index in system.parts and system.parts_columns
 2021-03-25 16:02:33 +03:00
Anton Popov
6a15431be7 Merge remote-tracking branch 'upstream/master' into HEAD 2021-03-25 15:57:35 +03:00
Mike Kot
285af08949 Merge remote-tracking branch 'upstream/master' into feature/attach-partition-local 2021-03-24 22:34:20 +03:00
alexey-milovidov
612d4fb073
Update IMergeTreeDataPart.cpp 2021-03-24 02:03:14 +03:00
Pavel Kovalenko
a92cf30b67 Code review fixes. 2021-03-23 13:33:07 +03:00
Alexey Milovidov
8d0210b510 Expose DateTime64 minmax part index in system.parts and system.parts_columns #18244 2021-03-23 01:16:41 +03:00
Mike Kot
c55a73b752 Added the solution to handle the corruption case 
When the part data (e.g. data.bin) is corrupted, but the checksums.txt
is present -- explicitly deleting the checksums.txt.

Removed the extra logging, changes some exceptions message.
 2021-03-22 17:23:43 +03:00
Anton Popov
173d2ea1f4 Merge remote-tracking branch 'upstream/master' into HEAD 2021-03-16 02:50:14 +03:00
Anton Popov
1b07d28043 fix unwanted changes 2021-03-13 02:59:42 +03:00
Anton Popov
bc417cf54a refactoring of serializations 2021-03-09 17:46:52 +03:00
alesapin
5b3161e0b5 Get rid of const_cast 2021-03-05 20:24:06 +03:00
Anton Ivashkin
d08b481660 Fixes by review responces 2021-03-05 19:20:38 +03:00
Anton Ivashkin
e69124a0a6 Merge master 2021-03-04 13:26:40 +03:00
alesapin
9ebf1b4fad Get rid of separate minmax index fields 2021-03-02 13:33:54 +03:00
Anton Ivashkin
c891cf4557 Fixes by review response 2021-02-26 12:48:57 +03:00
Anton Ivashkin
4d44d75bc7 Fix build after merge one more time 2021-02-08 14:45:10 +03:00
Anton Ivashkin
e64c63c611 Merge master 2021-02-05 20:10:06 +03:00
Anton Ivashkin
df6c882aab Fix build after merge 2021-02-05 18:52:40 +03:00
Anton Popov
a8f3078ce9 Merge remote-tracking branch 'upstream/master' into HEAD 2021-01-27 19:48:55 +03:00
Anton Popov
c7070da85a better abstractions in disk interface 2021-01-26 17:49:35 +03:00
Anton Popov
658f24dcff
Merge pull request #19358 from CurtizJ/fix-subcolumns 
Fix several cases, while reading subcolumns
 2021-01-25 20:26:07 +03:00
Azat Khuzhin
cb951c2116 Add metrics for MergeTree parts types 
- PartsWide
- PartsCompact
- PartsInMemory
 2021-01-21 21:17:00 +03:00
Anton Popov
ac3de63a71 fix several cases, while reading subcolumns 2021-01-21 15:34:11 +03:00
Anton Ivashkin
357d98eb36 Merge master 2021-01-20 12:23:03 +03:00
Anton Ivashkin
eba98b04b0 Zero copy replication over S3: Hybrid storage support 2021-01-18 19:16:45 +03:00
Alexey Milovidov
029302d766 Merge with master 2021-01-16 17:09:44 +03:00
Alexey Milovidov
24c8e53440 Merge branch 'master' into multiple-nested 2021-01-16 16:28:40 +03:00
alexey-milovidov
2e2988e5d8
Merge pull request #19146 from azat/server-memory-limit-blocking 
MemoryTracker: Do not ignore server memory limits during blocking by default
 2021-01-16 11:09:19 +03:00
alexey-milovidov
5f189c5756
Merge pull request #19122 from ClickHouse/data-part-better-code 
Add metrics for part number in MergeTree in ClickHouse
 2021-01-16 00:20:15 +03:00
Azat Khuzhin
61b2d0ce42 MemoryTracker: Do not ignore server memory limits during blocking by default 2021-01-15 22:46:58 +03:00
alexey-milovidov
b97beea22a
Merge pull request #19101 from ClickHouse/check_compression_codec_read 
Fix compression codec read for empty files
 2021-01-15 20:55:58 +03:00
Alexey Milovidov
e238fd64ac Add part metrics 2021-01-15 15:28:53 +03:00
Alexey Milovidov
6a2a5e53ed Slightly better code of IMergeTreeDataPart #18955 2021-01-15 15:15:13 +03:00
alesapin
e106df2ad0 Fix comment 2021-01-15 12:10:03 +03:00
alesapin
0662d6bd7d Fix compression codec read for empty files 2021-01-15 12:04:23 +03:00
Alexey Milovidov
8276a1c8d2 Faster parts removal, more safe and efficient interface of IDisk 2021-01-14 19:24:13 +03:00
Anton Popov
0e903552a0 fix TTLs with WHERE 2021-01-13 17:04:27 +03:00
Anton Popov
d7200ee2ed minor changes 2021-01-13 02:20:32 +03:00
Anton Popov
15ead18673 Merge remote-tracking branch 'upstream/master' into HEAD 2021-01-12 19:46:10 +03:00
Anton Popov
5822ee1f01 allow multiple rows TTL with WHERE expression 2021-01-12 02:07:21 +03:00
Anton Popov
36ae0e4d35 Merge remote-tracking branch 'upstream/master' into HEAD 2021-01-11 13:51:12 +03:00
Azat Khuzhin
b1f08f5c27 Rename FileSyncGuard to DirectorySyncGuard 2021-01-07 20:26:18 +03:00
Azat Khuzhin
513a824f30 Fix fsync_part_directory for parts renames 2021-01-07 19:30:25 +03:00
Alexey Milovidov
36e1361cf8 Miscellaneous 2021-01-07 15:29:34 +03:00
Anton Popov
11283e3d81 Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-25 21:25:59 +03:00
Anton Popov
b60c00ba74 refactoring of TTL stream 2020-12-25 18:46:13 +03:00
alesapin
54455b4740 Add test for already working code 2020-12-23 14:53:49 +03:00
Anton Popov
a42b00c9aa Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-17 20:43:23 +03:00
Anton Ivashkin
0f0500ca0c Merge master 2020-12-16 18:31:13 +03:00
alesapin
b307e545a9 Fix check 2020-12-09 14:46:04 +03:00
alesapin
0f4056fd95 Add additional size check in debug mode 2020-12-09 14:23:37 +03:00
Anton Popov
66e0add2ba fix nested 2020-12-07 16:35:12 +03:00
Anton Popov
b384beb564 Merge remote-tracking branch 'upstream/master' into HEAD 2020-11-23 17:46:51 +03:00
Nicolae Vartolomei
040aba9f85 Add uuid.txt to checksums for parts stored on disk 
We are breaking backwards compatibility anyway (but agted by a setting)
 2020-11-20 13:49:17 +00:00
Nicolae Vartolomei
425dc4b11b Add unique identifiers IMergeTreeDataPart structure 
For now uuids are not generated at all, they are present only if the
part is updated manually (as you can see in the integration test).

The only place where they can be seen today by an end user is in
`system.parts` table. I was looking for hiding this column behind an
option but couldn't find an easy way to do that.

Likely this is also required for WAL, but need to think how not to break
compatibility.

Relates to #13574, https://github.com/ClickHouse/ClickHouse/issues/13574

Next 1: In the upcoming PR the plan is to integrate de-duplication based on
these fingerprints in the query pipeline.

Next 2: We'll enable automatic generation of uuids and come up with a
way for conditionally sending uuids when processing distributed queries
only when part movement is in progress.
 2020-11-19 13:14:25 +00:00
Anton Popov
245c395a68 Merge remote-tracking branch 'upstream/master' into HEAD 2020-11-06 22:00:32 +03:00
Anton Ivashkin
1742fb3256 Merge master 2020-11-03 12:27:16 +03:00
Anton Ivashkin
78021714f1 S3 zero copy replication: more simple s3 check 2020-11-03 12:20:26 +03:00
Mikhail Filimonov
41971e073a
Fix typos reported by codespell 2020-10-27 12:04:03 +01:00
Anton Popov
a249f0c95e Merge remote-tracking branch 'upstream/master' into HEAD 2020-10-23 22:05:00 +03:00
Anton Ivashkin
e3879afa69 S3 zero copy replication: fix virtual method default parameter 2020-10-23 16:27:50 +03:00
Anton Ivashkin
652c56e74e Fix style, fix build 2020-10-22 16:07:20 +03:00
Anton Ivashkin
f833501c77 Merge branch 'master' into s3_zero_copy_replication 2020-10-22 11:17:21 +03:00
Alexey Milovidov
2619efadc8 Fix multiple issues with memory tracking 2020-10-21 03:31:34 +03:00
Vladimir Chebotarev
aa5f207fd4
Added disable_merges option for volumes in multi-disk configuration (#13956) 
Co-authored-by: Alexander Kazakov <Akazz@users.noreply.github.com>
 2020-10-20 18:10:24 +03:00
Anton Ivashkin
14a78f87b0 Zero copy replication over S3: fetch instead of merge 2020-10-15 19:14:47 +03:00
Pavel Kovalenko
ed61c5681b Use 'moving' directory instead of 'detached' when move part to another disk/volume. 2020-10-15 16:55:13 +03:00
Anton Popov
cbe12a532e allow to extract subcolumns from column 2020-10-13 22:39:22 +03:00
Anton Ivashkin
9272ed06b4 Move Zookeeper lock for S3 shared part in IMergeTreeDataPart 2020-10-09 17:24:10 +03:00
Anton Ivashkin
766dbfd2be Merge master 2020-10-09 15:00:14 +03:00
Anton Ivashkin
acf86568a7 S3 zero copy replication proof of concept 2020-10-08 18:45:10 +03:00
Pavel Kovalenko
69c126f1f1 Possibility to move part to another disk/volume if first attempt was failed. 2020-10-07 14:35:28 +03:00
Pavel Kovalenko
69b4bc6f33 Proper error handling during insert into MergeTree with S3. 2020-10-06 12:38:00 +03:00
Anton Popov
edc2fe2226 Merge remote-tracking branch 'upstream/master' into HEAD 2020-09-18 17:51:54 +03:00
Anton Popov
cb4801e3be allow to read subcolumns of complex types 2020-09-18 02:12:43 +03:00
Anton Popov
3b9be70c8c Merge remote-tracking branch 'upstream/master' into HEAD 2020-09-05 18:53:55 +03:00
alesapin
acc0ee0657 Apply TTL if it's not calculated for part 2020-09-03 11:59:41 +03:00
Anton Popov
4fe57fc169 Merge remote-tracking branch 'upstream/master' into HEAD 2020-09-02 01:29:44 +03:00
alesapin
067eb4599d Fix empty columns case 2020-08-31 16:39:27 +03:00
alesapin
77faf9587f Better interface 2020-08-28 12:07:20 +03:00
alesapin
d426751e26 Default codec if part is not stored on disk 2020-08-27 16:32:23 +03:00
alesapin
1225c09a02 Fix default codec for in-memory parts 2020-08-27 11:35:55 +03:00
alesapin
44eb702fbe Remove redundant code 2020-08-26 22:41:57 +03:00
alesapin
3a680dc8fb Fix removal bug 2020-08-26 22:40:04 +03:00
alesapin
1fb93a8bf2 Remove read helpers 2020-08-26 18:57:06 +03:00
alesapin
18eb0dbcaa Better remove 2020-08-26 18:39:11 +03:00
alesapin
2fc80189af Add default compression codec to merge tree data part 2020-08-26 18:29:46 +03:00
Anton Popov
4df5c72878 Merge remote-tracking branch 'upstream/master' into HEAD 2020-08-24 12:57:27 +03:00
alexey-milovidov
7b53a0ef33 Revert "Added allow_merges option for volumes in multi-disk configuration (#13402)" 
This reverts commit 1e2616542a.
 2020-08-21 18:44:29 +03:00
Vladimir Chebotarev
1e2616542a
Added allow_merges option for volumes in multi-disk configuration (#13402) 2020-08-21 12:04:13 +03:00
alesapin
1707f84a44 Less strict check and rare rename bug 2020-07-27 12:42:37 +03:00
alesapin
78d357f0d2 Add a test for sticking mutations bug 2020-07-25 17:42:20 +03:00
alesapin
b3ec3b1ed1 Fix loading parts without checksums 2020-07-16 13:54:49 +03:00
Anton Popov
ca346ea13c rename fsync-related settings 2020-07-03 02:41:37 +03:00
Anton Popov
53e955c6dd several fixes 2020-06-29 23:36:18 +03:00
Anton Popov
b2aa565a37 allow to turn on fsync on inserts, merges and fetches 2020-06-27 17:52:16 +03:00
alesapin
6f1824f0ea Correct merge with master 2020-06-26 14:30:23 +03:00
alesapin
e9c47dc89c Merge branch 'master' into CurtizJ-polymorphic-parts 2020-06-26 14:27:19 +03:00
alesapin
dffdece350 getColumns in StorageInMemoryMetadta (only compilable) 2020-06-17 19:39:58 +03:00
alesapin
1afdebeebd Primary key in storage metadata 2020-06-17 15:39:20 +03:00
alesapin
ab61abccc1 Partition key in StorageInMemoryMetadata 2020-06-17 13:34:23 +03:00
Anton Popov
df3dfd5b81 fix clang-tidy build 2020-06-04 01:00:02 +03:00
Anton Popov
11c4e9dde3 in-memory parts: fix 'check' query 2020-06-03 22:19:49 +03:00
Anton Popov
1ce09e1faa Merge remote-tracking branch 'upstream/master' into polymorphic-parts 2020-06-03 16:27:54 +03:00
Anton Popov
caf1e4e8cc in-memory-parts: fixes 2020-06-03 12:51:23 +03:00
Alexey Milovidov
b78c48310b Whitespaces 2020-05-30 01:37:44 +03:00
Anton Popov
c919840722 in-memory parts: partition commands 2020-05-29 18:02:12 +03:00
Anton Popov
d8342e5b12 in-memory parts: better restore and clear stale wal files 2020-05-27 23:05:55 +03:00
alesapin
c7cda399c7 Merge branch 'master' into consistent_metadata2 2020-05-27 12:51:56 +03:00
alesapin
8614487a5a Fix primary key check 2020-05-26 16:46:19 +03:00
Alexey Milovidov
c858f4d89c Clarifications 2020-05-26 02:47:21 +03:00
alesapin
c3a6571036 Compilable code 2020-05-25 20:22:20 +03:00
alesapin
6281dd6893
Merge pull request #11115 from ClickHouse/consistent_metadata 
Refactoring in storage metadata (more consistent keys)
 2020-05-25 13:09:56 +03:00
Alexey Milovidov
7e1813825b Return old names of macros 2020-05-24 01:24:01 +03:00
Alexey Milovidov
e1695feb7f Apply all transformations again 2020-05-23 23:40:32 +03:00
Alexey Milovidov
85f84550ba Progress on task 2020-05-23 23:37:37 +03:00
Alexey Milovidov
7e2fb9ad65 Apply all transformations again 2020-05-23 22:38:30 +03:00
Alexey Milovidov
eacff92d0e Progress on task 2020-05-23 22:35:08 +03:00
Alexey Milovidov
29762240de Remove duplicate whitespaces (preparation) 2020-05-23 22:31:54 +03:00
Alexey Milovidov
9d2a0d2dd7 Apply all transformations again 2020-05-23 21:59:49 +03:00
Alexey Milovidov
a2ad11897f Remove duplicate whitespaces (preparation) 2020-05-23 21:53:58 +03:00
Alexey Milovidov
1f13515a65 Make all LOG in single line (preparation) 2020-05-23 21:31:37 +03:00
Alexey Milovidov
8042e5febe find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}\5{}", \4, \6);/' 2020-05-23 19:58:15 +03:00
Alexey Milovidov
ee4ffbc332 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}", \4);/' 2020-05-23 19:47:56 +03:00
alesapin
327d17ac6a Better 2020-05-21 22:46:03 +03:00
alesapin
616902a995 Sorting and primary key (broken) 2020-05-20 21:11:38 +03:00
alesapin
8bc527eecd Compile partition key 2020-05-20 15:16:55 +03:00
Gleb Novikov
1a25ac6e1f Merge branch 'master' into refactor-reservations 2020-05-16 23:34:45 +03:00
alesapin
26606dd640 Fix polymorphic parts 2020-05-15 13:36:35 +03:00
Gleb Novikov
390b39b272 VolumePtr instead of DiskPtr in MergeTreeData* 2020-05-10 00:24:15 +03:00
Anton Popov
aa723d245f Merge remote-tracking branch 'upstream/master' into polymorphic-parts 2020-04-30 18:27:39 +03:00
Anton Popov
4069dbcc58 in-memory parts: add waiting for insert 2020-04-20 04:38:38 +03:00
Anton Popov
391f7c34be in memory parts: basic read/write 2020-04-17 20:30:46 +03:00
Alexey Milovidov
f704c465a7 Make aware of one special case of imcompletely written parts #9940 2020-04-16 00:52:21 +03:00
Anton Popov
79024d73a2 improve performance of index analysis with monotonic functions 2020-04-06 13:37:34 +03:00
Ivan Lezhankin
06446b4f08 dbms/ → src/ 2020-04-03 18:14:31 +03:00