Alexander Gololobov
75152ddabb
Apply filter only if needed
2022-12-28 18:07:35 +01:00
Alexander Gololobov
a18850458c
Test applying current filter at each step
2022-12-28 18:07:35 +01:00
Alexander Gololobov
29b5c4af07
Test dirty intermediate changes
2022-12-28 18:07:35 +01:00
Alexander Gololobov
c561acb774
Properly handle low cardinality column as prewhere filter
2022-12-28 18:07:35 +01:00
Alexander Gololobov
aa276b230b
Don't need to save filter and rows_per_granule from previous step
2022-12-28 18:07:35 +01:00
Alexander Gololobov
c4a01cbd5b
Fix for propely cleaning rows_per_granule_original between prewhere steps
2022-12-28 18:07:35 +01:00
Alexander Gololobov
abbb58107c
Fix for "out of bound" in ColumnVector::insertRangeFrom called from shrink()
2022-12-28 18:07:35 +01:00
Alexander Gololobov
79874e8733
Fix for "Invalid number of rows in Chunk"
2022-12-28 18:07:35 +01:00
Alexander Gololobov
bdf51545f7
Added FilterWithCachedCount class instead of caching counts in filter_bytes_map
2022-12-28 18:07:35 +01:00
Azat Khuzhin
31a88d4eae
Fix PREWHERE with row-level filters (when row filter is always true/false)
...
In case of row-level filters optimized out, i.e. converted to
always true/false, it is possible for MergeTreeRangeReader to reuse
incorrect statistics for the filter (countBytesInResultFilter()), and
because of this it simply does not apply other filters, since it assume
that this filter does not need to filter anything.
Fixes : #40956
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-28 18:07:35 +01:00
Alexander Gololobov
d44392b366
Checking the fix for "Invalid number of rows in Chunk"
2022-12-28 18:07:35 +01:00
kssenii
83514fa2ef
Refactor
2022-09-05 20:08:22 +02:00
Robert Schulze
a7734672b9
Use std::popcount, ::countl_zero, ::countr_zero functions
...
- Introduced with the C++20 <bit> header
- The problem with __builtin_c(l|t)z() is that 0 as input has an
undefined result (*) and the code did not always check. The std::
versions do not have this issue.
- In some cases, we continue to use buildin_c(l|t)z(), (e.g. in
src/Common/BitHelpers.h) because the std:: versions only accept
unsigned inputs (and they also check that) and the casting would be
ugly.
(*) https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html
2022-07-31 15:16:51 +00:00
Alexander Gololobov
48de02a7b8
Capitalized const name
2022-07-25 16:32:16 +02:00
Alexander Gololobov
594195451e
Cleanups
2022-07-24 12:21:18 +02:00
Alexander Gololobov
1ea9f143ff
Leave only _row_exists-based implementation of lightweight delete
2022-07-21 11:26:13 +02:00
Alexander Gololobov
ae0d00083c
Renamed __row_exists to _row_exists
2022-07-18 20:07:36 +02:00
Alexander Gololobov
f324ca9921
Cleanups
2022-07-18 20:07:22 +02:00
Alexander Gololobov
9de72d995a
POC lightweight delete using __row_exists virtual column and prewhere-like filtering
2022-07-18 20:06:42 +02:00
jianmei zhang
ca42f649da
Rewrite logic for loading deleted mask related to getDeletedMask()
2022-07-15 15:31:10 +08:00
jianmei zhang
d37152a5d6
Remove loadDeletedMask() and get deleted mask when needed
2022-07-15 12:32:42 +08:00
jianmei zhang
7e433859ea
Change deleted rows mask from String to Native UInt8 format
2022-07-15 12:32:41 +08:00
jianmei zhang
8ad2bb7c33
Code changes due to master new fixes, and update reference for mutations table
2022-07-15 12:32:41 +08:00
jianmei zhang
11fdea6e4b
Add missing code for deleted_mask_filter_holder
2022-07-15 12:32:41 +08:00
jianmei zhang
9d27af7ee2
For some columns mutations, skip to apply deleted mask when read some columns. Also add unit test case
2022-07-15 12:32:41 +08:00
jianmei zhang
b4a37e1e22
Disable optimizations for count() when lightweight delete exists, add hasLightweightDelete() function in IMergeTreeDataPart
2022-07-15 12:32:41 +08:00
jianmei zhang
8696319d62
Support lightweight delete execution using string as deleted rows mask,also part of select can handle LWD
2022-07-15 12:32:41 +08:00
Alexander Gololobov
0ee47363d4
Fixed includes
2022-06-22 19:08:18 +02:00
Alexander Gololobov
e5b55b965b
Removed incorrect check
2022-06-22 17:23:09 +02:00
Alexander Gololobov
dbc6d1a159
Cleanups
2022-06-22 17:23:09 +02:00
Alexander Gololobov
b3922461b3
Properly handle empty actions
2022-06-22 17:23:09 +02:00
Alexander Gololobov
ba89c3954c
Do not add the same vitrual if it has been added by prev_reader
2022-06-22 17:23:09 +02:00
Alexander Gololobov
a9e3b8d29e
Don't read the same columns again
2022-06-22 17:23:09 +02:00
Alexander Gololobov
4e426c63cc
Debuging test failures
2022-06-22 17:23:09 +02:00
Alexander Gololobov
6a26325fab
Test dirty hacks for multiple PREWHERE steps
2022-06-22 17:23:05 +02:00
Alexander Gololobov
87b669f439
Intermediate changes
2022-06-22 17:17:42 +02:00
Alexander Gololobov
64a2f3734b
Protect ReadResult internals from MergeTreeRangeReader clients
2022-06-22 17:17:42 +02:00
Danila Kutenin
08e3f77a9c
Optimize most important parts with NEON SIMD
...
First part, updated most UTF8, hashing, memory and codecs. Except
utf8lower and upper, maybe a little later.
That includes huge amount of research with movemask dealing. Exact
details and blog post TBD.
2022-06-15 13:19:29 +00:00
Alexey Milovidov
f860407af5
Merge pull request #37777 from ClickHouse/avx512_tail_zero
...
Use multitarget framework for numZerosInTail implementation
2022-06-04 01:46:56 +03:00
Alexander Gololobov
0b004b907a
Added AVX2 implementation of numZerosInTail
2022-06-02 13:01:13 +02:00
Nikolai Kochetov
fc726c236f
Merge pull request #37661 from ClickHouse/no_alias_actions
...
Trying to remove prewhere_info.alias_actions as they seem to always be included in prewhere_actions
2022-06-02 12:31:04 +02:00
Alexander Gololobov
d8e4dcf9e2
Use multitarget framework for numZerosInTail implementation
2022-06-02 11:23:44 +02:00
Alexander Gololobov
a0cf902d49
Merge pull request #37588 from yaqi-zhao/avx512_tail_zero
...
add avx512 support for mergetree reader
2022-06-02 08:51:52 +03:00
Alexander Gololobov
26609a1875
Style fixes
2022-05-31 21:41:10 +02:00
yaqi-zhao
a2857491c4
add avx512 support for mergetreereader
2022-05-30 20:53:00 -04:00
Alexander Gololobov
e2dd6f6249
Removed prewhere_info.alias_actions
2022-05-30 19:58:23 +02:00
Alexander Gololobov
c484a1c269
Preserve order of virtual columns in the header and in data chunks when adding _part_offset together with other virtual columns
2022-04-25 12:10:56 +02:00
roverxu
29a842bf22
feat(...): [LWD] support getting _part_offset of a row
2022-03-15 15:40:10 +08:00
Maksim Kita
b1a956c5f1
clang-tidy check performance-move-const-arg fix
2022-03-02 18:15:27 +00:00
Amos Bird
d3bd8b5f93
Cosmetic fix
2022-02-17 14:31:22 +08:00
