* feat: Preserve number of streams after evaluation the window functions to allow parallel stream processing
* fix style
* fix style
* fix style
* setting query_plan_preserve_num_streams_after_window_functions default true
* fix tests by SETTINGS query_plan_preserve_num_streams_after_window_functions=0
* fix test references
* Resize the streams after the last window function, to keep the order between WindowTransforms (and WindowTransform works on single stream anyway).
* feat: Preserve number of streams after evaluation the window functions to allow parallel stream processing
* fix style
* fix style
* fix style
* setting query_plan_preserve_num_streams_after_window_functions default true
* fix tests by SETTINGS query_plan_preserve_num_streams_after_window_functions=0
* fix test references
* Resize the streams after the last window function, to keep the order between WindowTransforms (and WindowTransform works on single stream anyway).
* add perf test
* perf: change the dataset from 50M to 5M
* rename query_plan_preserve_num_streams_after_window_functions -> query_plan_enable_multithreading_after_window_functions
* update test reference
* fix clang-tidy
---------
Co-authored-by: Nikita Taranov <nikita.taranov@clickhouse.com>
* Release more num_streams if data is small
Besides the sum_marks and min_marks_for_concurrent_read, we could also involve the
system cores to get the num_streams if the data is small. Increasing the num_streams
and decreasing the min_marks_for_concurrent_read would improve the parallel performance
if the system has plentiful cores.
Test the patch on 2x80 vCPUs system. Q39 of clickbench has got 3.3x performance improvement.
Q36 has got 2.6x performance improvement. The overall geomean has got 9% gain.
Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>
* Release more num_streams if data is small
Change the min marks from 4 to 8 as the profit is small and 8 granules
is the default block size.
Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>
---------
Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>
The problem was the order of the columns, in case of SELECT FINAL it got
"counters_Map.count", "counters_Map.id"
But in case of OPTIMIZE FINAL it got "counters_Map.id",
"counters_Map.count" correctly.
Note, that this bugs exists there from the very recent versions, I've
checked 19.x and it was there.
P.S. there is a workaround for this problem, if you will use one of the
following patterns for key columns:
- *ID
- *Key
- *Type
That way it will be explicitly matched as key and everything will work.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
