ClickHouse/src/Processors
Jiebin Sun df17cd467b
Release more num_streams if data is small (#53867)
* Release more num_streams if data is small

Besides the sum_marks and min_marks_for_concurrent_read, we could also involve the
system cores to get the num_streams if the data is small. Increasing the num_streams
and decreasing the min_marks_for_concurrent_read would improve the parallel performance
if the system has plentiful cores.

Test the patch on 2x80 vCPUs system. Q39 of clickbench has got 3.3x performance improvement.
Q36 has got 2.6x performance improvement. The overall geomean has got 9% gain.

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>

* Release more num_streams if data is small
Change the min marks from 4 to 8 as the profit is small and 8 granules
is the default block size.

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>

---------

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>
2023-10-16 18:41:38 +02:00
..
examples
Executors Revert "Revert "Add settings for real-time updates during query execution"" 2023-09-09 12:29:39 +02:00
Formats Fix some typos 2023-10-15 15:43:02 +02:00
Merges Print more information about one logical error in MergeTreeDataWriter (#55001) 2023-09-26 13:23:26 -07:00
QueryPlan Release more num_streams if data is small (#53867) 2023-10-16 18:41:38 +02:00
Sinks Fixing PR comments 2023-09-19 12:15:01 +02:00
Sources Merge pull request #55419 from evillique/mongo-fix 2023-10-10 01:30:04 +02:00
tests
Transforms Merge pull request #55489 from kitaisreal/external-aggregation-improve-performance 2023-10-12 18:00:41 +02:00
TTL
Chunk.cpp Revert "Revert "Add settings for real-time updates during query execution"" 2023-09-09 12:29:39 +02:00
Chunk.h Apply suggestions from code review 2023-08-22 19:05:06 +02:00
CMakeLists.txt
ConcatProcessor.cpp
ConcatProcessor.h Make some classes 'final' 2023-02-02 15:59:39 +00:00
DelayedPortsProcessor.cpp
DelayedPortsProcessor.h
ForkProcessor.cpp
ForkProcessor.h
IAccumulatingTransform.cpp
IAccumulatingTransform.h
IInflatingTransform.cpp
IInflatingTransform.h
IProcessor.cpp Revert "Revert "Add settings for real-time updates during query execution"" 2023-09-09 12:29:39 +02:00
IProcessor.h Revert "Revert "Add settings for real-time updates during query execution"" 2023-09-09 12:29:39 +02:00
ISimpleTransform.cpp
ISimpleTransform.h
ISink.cpp
ISink.h
ISource.cpp Fix summary reporting with parallel replicas with LIMIT (#53050) 2023-09-04 15:53:06 +02:00
ISource.h Fix summary reporting with parallel replicas with LIMIT (#53050) 2023-09-04 15:53:06 +02:00
LimitTransform.cpp Revert "Revert "Add settings for real-time updates during query execution"" 2023-09-09 12:29:39 +02:00
LimitTransform.h Increase amount of rows in requests for partial result tests 2023-09-09 10:33:05 +00:00
OffsetTransform.cpp
OffsetTransform.h
PingPongProcessor.cpp
PingPongProcessor.h
Port.cpp
Port.h
QueueBuffer.h
ResizeProcessor.cpp Merge branch 'master' into not-ready-set-bug 2023-08-21 10:59:51 +00:00
ResizeProcessor.h
RowsBeforeLimitCounter.h