Commit Graph

314 Commits

Author SHA1 Message Date
kssenii
2a631aaf08 Final fixes 2021-05-29 00:34:44 +03:00
Nikolai Kochetov
58fbc544cc Add more comments. 2021-05-28 20:16:09 +03:00
Nikolai Kochetov
295a302bc8 Remove settings from ReadFromMergeTree. 2021-05-28 17:34:02 +03:00
Nikolai Kochetov
91e8c2c75d Remove MergingFinal step. 2021-05-28 12:52:33 +03:00
Nikolai Kochetov
94f1ac5a16 Remove some commented code. 2021-05-28 12:41:07 +03:00
Nikolai Kochetov
c80ab6166a Fix style. 2021-05-28 12:26:44 +03:00
Nikolai Kochetov
4e28b7cb02 Fix _sample_factor column. 2021-05-28 12:23:46 +03:00
Nikolai Kochetov
936bd8a011 Merge branch 'master' into refactor-MergeTreeDataSelectExecutor 2021-05-27 20:47:39 +03:00
Nikolai Kochetov
1aeb705b20 Fix some tests. 2021-05-27 19:53:58 +03:00
Nikolai Kochetov
cbdf3752ef Part 3. 2021-05-27 16:40:33 +03:00
kssenii
73f16ee9ee Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-26 23:08:08 +03:00
Nikolai Kochetov
a51a6ea0b7 Part 2. 2021-05-26 21:14:43 +03:00
kssenii
940eab6924 Review fixes 2021-05-26 09:26:19 +03:00
Nikolai Kochetov
34eaa48294 Part 1. 2021-05-25 19:34:43 +03:00
kssenii
c3fc2ff04b Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into modulo-partition-key 2021-05-24 22:08:49 +00:00
Alexey Milovidov
d32819f068 Mark false positives for PVS-Studio 2021-05-24 06:59:12 +03:00
kssenii
659e57e60f Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into modulo-partition-key 2021-05-23 22:46:00 +00:00
kssenii
bbf56e35ac Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-23 10:54:48 +03:00
Nikita Vasilev
dd343f087e fix 2021-05-21 19:23:33 +03:00
kssenii
94b5763dbd Revert "Different approach to fix modulo"
This reverts commit 665f1aa0f6.
2021-05-21 16:14:01 +00:00
kssenii
665f1aa0f6 Different approach to fix modulo 2021-05-21 14:30:08 +00:00
kssenii
bd0bc8e3e7 Revert "Adjust minMax index"
This reverts commit c69f587f8a.
2021-05-21 12:01:08 +00:00
Nikita Vasilev
135b0ed69d fix 2021-05-20 23:27:32 +03:00
Nikita Vasilev
9f2bcacb46 Merge remote-tracking branch 'upstream/master' into nikvas0/simple_optimizations 2021-05-20 19:34:11 +03:00
kssenii
c69f587f8a Adjust minMax index 2021-05-20 15:32:18 +00:00
kssenii
477214a57f Fix partitionPrunner with modulo 2021-05-19 17:36:32 +00:00
alesapin
22cc1c953f Revert "PVS Studio fixes"
This reverts commit 87c6e39a29.
2021-05-18 10:34:16 +03:00
alesapin
87c6e39a29 PVS Studio fixes 2021-05-17 23:23:32 +03:00
kssenii
9b8df78fdd Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-17 17:42:05 +03:00
alesapin
67d34c0136 merge with master 2021-05-17 14:13:18 +03:00
alesapin
17f229857c Merge branch 'master' into nvartolomei-parts-move 2021-05-17 13:52:48 +03:00
Nikita Vasilev
6102ea643a Merge remote-tracking branch 'upstream/master' into nikvas0/simple_optimizations 2021-05-16 19:51:41 +03:00
Amos Bird
07b1be5a76
Fix distributed processing when using projection 2021-05-16 22:40:06 +08:00
Amos Bird
a113acc40c
Fix empty key 2021-05-14 22:54:14 +08:00
kssenii
0527f0ea33 Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-12 16:54:18 +03:00
kssenii
393b026f2e Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-11 16:49:23 +03:00
alesapin
46e136b5c4
Merge branch 'master' into nv/parts-uuid-move-shard 2021-05-11 15:36:40 +03:00
Nikolai Kochetov
a7d717d767 Remove unused code. 2021-05-11 14:59:56 +03:00
Amos Bird
074c2ead9c
Fix 2021-05-11 18:22:23 +08:00
Amos Bird
5e9fab6d10
Comment debug info out 2021-05-11 18:12:29 +08:00
Amos Bird
5836e2e277
Fix issues in normal projection 2021-05-11 18:12:29 +08:00
Amos Bird
51d685bb9c
Even more fixes 2021-05-11 18:12:29 +08:00
Amos Bird
ddd4256a15
More fixes 2021-05-11 18:12:28 +08:00
Amos Bird
718c284437
Fix more tests 2021-05-11 18:12:28 +08:00
Amos Bird
ba17acbd63
Fix tests 2021-05-11 18:12:28 +08:00
Amos Bird
a3fd0b6f2e
Fix tests 2021-05-11 18:12:28 +08:00
Amos Bird
42f161b1dc
Fix more 2021-05-11 18:12:28 +08:00
Amos Bird
ef918aeaff
Rebase and fix 2021-05-11 18:12:28 +08:00
Amos Bird
9c069ebdbf
support prewhere, row_filter, read_in_order and decent projection selection
TODO set index analysis in projection
2021-05-11 18:12:27 +08:00
Amos Bird
35961c0c5d
Use ActionsDAG to rewrite projection queries 2021-05-11 18:12:27 +08:00
Nikolai Kochetov
29d7038e30
Add ProjectionDescription::Type enum. Remove some checks. 2021-05-11 18:12:27 +08:00
Amos Bird
ebaf42a448
Reformat and fix some tests 2021-05-11 18:12:27 +08:00
Nikolai Kochetov
672cfedd13
Disable normal projection by the number of granules. 2021-05-11 18:12:26 +08:00
Nikolai Kochetov
95431168c2
Try to enable normal projections. 2021-05-11 18:12:26 +08:00
Nikolai Kochetov
427f444ba4
Fix prewhere. 2021-05-11 18:12:26 +08:00
Nikolai Kochetov
3d36fb57eb
Cleanup code. 2021-05-11 18:12:26 +08:00
Nikolai Kochetov
3296c9292f
Try to merge projectons faster. 2021-05-11 18:12:26 +08:00
Amos Bird
264cff6415
Projections
TODO (suggested by Nikolai)

1. Build query plan fro current query (inside storage::read) up to WithMergableState
2. Check, that plan is simple enough: Aggregating - Expression - Filter - ReadFromStorage (or simplier)
3. Check, that filter is the same as filter in projection, and also expression calculates the same aggregation keys as in projection
4. Return WithMergableState if projection applies

3 will be easier to do with ActionsDAG, cause it sees all functions, and dependencies are direct (but it is possible with ExpressionActions also)

Also need to figure out how prewhere works for projections, and
row_filter_policies.

wip
2021-05-11 18:12:23 +08:00
Kruglov Pavel
49e7ed6e60
Merge pull request #23673 from amosbird/partitionvalue
Add _partition_value virtual column
2021-05-11 11:23:00 +03:00
kssenii
02288359c5 Less manual concatenation of paths 2021-05-08 13:59:55 +03:00
Nikita Vasilev
4bf5547350 fix 2021-05-06 11:29:24 +03:00
Nikita Vasilev
496d8ff46c improvements 2021-05-04 21:43:58 +03:00
Alexey Milovidov
02ce9cc725 Untangle UUID 2021-05-04 01:46:51 +03:00
Nikita Vasilev
564a484642 fix 2021-05-02 22:16:40 +03:00
kssenii
deb4903af8 Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-04-28 20:57:13 +03:00
kssenii
eeb71672a0 Change in Storages/* 2021-04-27 16:49:37 +03:00
Nicolae Vartolomei
1fa5871ff7 Fix bad rebase and introduce part_moves_between_shards_enable setting 2021-04-27 14:20:13 +01:00
Nicolae Vartolomei
53d57ffb52 Part movement between shards
Integrate query deduplication from #17348
2021-04-27 14:20:12 +01:00
Amos Bird
8a3b5c1fab
Add _partition_value virtual column 2021-04-27 16:15:59 +08:00
mergify[bot]
a5da512e39
Merge branch 'master' into pid 2021-04-24 00:37:39 +00:00
Amos Bird
50f2e488bd
Fix invalid virtual column expr 2021-04-21 10:29:03 +08:00
Amos Bird
32c84f77c3
Resurrect indexHint function. 2021-04-20 19:27:23 +08:00
Nikolai Kochetov
8d8e57615c A little bit better index description. 2021-04-16 12:42:23 +03:00
Nikolai Kochetov
be52b2889a Better description for key condition. 2021-04-15 20:30:04 +03:00
Nikolai Kochetov
f6d86d6032 Merge branch 'master' into add-read-from-mt-step 2021-04-12 15:23:32 +03:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297)
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
2021-04-11 02:33:54 +03:00
Nikolai Kochetov
28ca191102
Merge pull request #22087 from ClickHouse/better-filter-push-down
Better filter push down
2021-04-09 10:22:17 +03:00
Nikolai Kochetov
794d7c89b6 Remove ReverseRowsStep 2021-04-08 16:18:17 +03:00
Nikolai Kochetov
febb187da0 Added test, 2021-04-08 15:54:27 +03:00
Nikolai Kochetov
7ffbeac9df Add info about indexes to ReadFromMergeTree step. 2021-04-08 14:48:54 +03:00
Nikolai Kochetov
7c5a9133df Add index info to ReadFromStorageStep. 2021-04-08 11:19:04 +03:00
Azat Khuzhin
f157278b72 Safer SCOPE_EXIT
It executes the code in the dtor, that should never throw.
2021-04-06 21:10:37 +03:00
Nikolai Kochetov
9f39f5d52d Add more counters to MergeTreeDataSelectExecutor 2021-04-06 15:39:55 +03:00
Nikolai Kochetov
6102652c99 Merge branch 'master' into better-filter-push-down 2021-04-06 13:38:03 +03:00
Nikolai Kochetov
8fad179800 Fix some tests. 2021-03-30 18:14:20 +03:00
Nikolai Kochetov
88bfa6c9ba Add ReadFromMergeTree step. 2021-03-30 13:25:26 +03:00
Alexey Milovidov
230b2930b7 Merge branch 'master' into virtualutil 2021-03-30 00:43:00 +03:00
Alexey Milovidov
2a8ac01cdb Rename as suggested by Kita 2021-03-28 22:24:28 +03:00
Alexey Milovidov
50f712e198 Integrate mmap cache to the infrastructure 2021-03-28 04:10:30 +03:00
feng lv
b7622868fc remove useless code 2021-03-25 11:10:41 +00:00
Nikolai Kochetov
0e2f52518f Extract converting from UnionStep. 2021-03-25 12:57:14 +03:00
Nikolai Kochetov
a669f7d641 Merge branch 'master' into refactor-actions-dag 2021-03-05 18:21:14 +03:00
Nikolai Kochetov
9a39459888 Refactor ActionsDAG 2021-03-04 20:38:12 +03:00
Amos Bird
909cb3c243
Fix again.... 2021-03-04 22:27:07 +08:00
Amos Bird
fac832227c
Fix again 2021-03-04 19:43:03 +08:00
Amos Bird
9205fad8c7
Better 2021-03-04 19:43:03 +08:00
Amos Bird
4c9b0f666c
Fix error 2021-03-04 19:43:03 +08:00
Amos Bird
93b661ad5a
partition id pruning 2021-03-04 19:43:03 +08:00
Nikolai Kochetov
15921fbfcb Remove index by name from ActionsDAG 2021-03-03 23:01:07 +03:00
alesapin
c29d7c7f49 Shutup clang tidy 2021-03-02 19:13:36 +03:00