Commit Graph

31 Commits

Author SHA1 Message Date
alesapin
42b2383e01 Basic throttler implementation 2021-05-26 23:37:44 +03:00
kssenii
94b5763dbd Revert "Different approach to fix modulo"
This reverts commit 665f1aa0f6.
2021-05-21 16:14:01 +00:00
kssenii
665f1aa0f6 Different approach to fix modulo 2021-05-21 14:30:08 +00:00
kssenii
c7f51fe60e Modulo adjustment for partition key 2021-05-15 19:51:02 +00:00
Amos Bird
a3fd0b6f2e
Fix tests 2021-05-11 18:12:28 +08:00
Amos Bird
264cff6415
Projections
TODO (suggested by Nikolai)

1. Build query plan fro current query (inside storage::read) up to WithMergableState
2. Check, that plan is simple enough: Aggregating - Expression - Filter - ReadFromStorage (or simplier)
3. Check, that filter is the same as filter in projection, and also expression calculates the same aggregation keys as in projection
4. Return WithMergableState if projection applies

3 will be easier to do with ActionsDAG, cause it sees all functions, and dependencies are direct (but it is possible with ExpressionActions also)

Also need to figure out how prewhere works for projections, and
row_filter_policies.

wip
2021-05-11 18:12:23 +08:00
Anton Ivashkin
ace628d565 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into s3_zero_copy_replication 2021-03-12 12:58:32 +03:00
Amos Bird
e6522e1ebe
JBOD data balancer 2021-03-08 11:10:35 +08:00
Anton Ivashkin
5b267b7eec Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into s3_zero_copy_replication 2021-02-26 13:15:18 +03:00
Anton Ivashkin
c891cf4557 Fixes by review response 2021-02-26 12:48:57 +03:00
Ivan
414f470c79
Make Poco HTTP Server zero-copy again (#19516)
* Refactoring: part 1

* Refactoring: part 2

* Handle request using ReadBuffer interface

* Struggles with ReadBuffer's

* Fix URI parsing

* Implement parsing of multipart/form-data

* Check HTTP_LENGTH_REQUIRED before eof() or will hang

* Fix HTTPChunkedReadBuffer

* Fix build and style

* Fix test

* Resist double-eof

* Fix arcadian build
2021-02-19 15:51:26 +03:00
Anton Ivashkin
eba98b04b0 Zero copy replication over S3: Hybrid storage support 2021-01-18 19:16:45 +03:00
Anton Ivashkin
0f0500ca0c Merge master 2020-12-16 18:31:13 +03:00
Nicolae Vartolomei
040aba9f85 Add uuid.txt to checksums for parts stored on disk
We are breaking backwards compatibility anyway (but agted by a setting)
2020-11-20 13:49:17 +00:00
Nicolae Vartolomei
746f8e45f5 All new parts must have uuids 2020-11-19 13:18:03 +00:00
Nicolae Vartolomei
425dc4b11b Add unique identifiers IMergeTreeDataPart structure
For now uuids are not generated at all, they are present only if the
part is updated manually (as you can see in the integration test).

The only place where they can be seen today by an end user is in
`system.parts` table. I was looking for hiding this column behind an
option but couldn't find an easy way to do that.

Likely this is also required for WAL, but need to think how not to break
compatibility.

Relates to #13574, https://github.com/ClickHouse/ClickHouse/issues/13574

Next 1: In the upcoming PR the plan is to integrate de-duplication based on
these fingerprints in the query pipeline.

Next 2: We'll enable automatic generation of uuids and come up with a
way for conditionally sending uuids when processing distributed queries
only when part movement is in progress.
2020-11-19 13:14:25 +00:00
Anton Ivashkin
e3879afa69 S3 zero copy replication: fix virtual method default parameter 2020-10-23 16:27:50 +03:00
Anton Ivashkin
652c56e74e Fix style, fix build 2020-10-22 16:07:20 +03:00
Anton Ivashkin
b877459cf7 Zero copy replication over S3: check s3 storage 2020-10-14 18:05:59 +03:00
Anton Ivashkin
acf86568a7 S3 zero copy replication proof of concept 2020-10-08 18:45:10 +03:00
Anton Popov
4fe57fc169 Merge remote-tracking branch 'upstream/master' into HEAD 2020-09-02 01:29:44 +03:00
alesapin
2fc80189af Add default compression codec to merge tree data part 2020-08-26 18:29:46 +03:00
Anton Popov
4df5c72878 Merge remote-tracking branch 'upstream/master' into HEAD 2020-08-24 12:57:27 +03:00
Anton Popov
b2aa565a37 allow to turn on fsync on inserts, merges and fetches 2020-06-27 17:52:16 +03:00
alesapin
6f1824f0ea Correct merge with master 2020-06-26 14:30:23 +03:00
Anton Popov
1ce09e1faa Merge remote-tracking branch 'upstream/master' into polymorphic-parts 2020-06-03 16:27:54 +03:00
Alexey Milovidov
25f941020b Remove namespace pollution 2020-05-31 00:57:37 +03:00
Anton Popov
d8342e5b12 in-memory parts: better restore and clear stale wal files 2020-05-27 23:05:55 +03:00
Anton Popov
4c03f48703 in-memory parts: better restoring 2020-05-15 03:39:15 +03:00
Anton Popov
42997bce86 im-memory parts: replication 2020-04-29 20:14:49 +03:00
Ivan Lezhankin
06446b4f08 dbms/ → src/ 2020-04-03 18:14:31 +03:00