Commit Graph

271 Commits

Author SHA1 Message Date
Vitaly Baranov
92bf2cfb50
Merge pull request #20968 from vitlibar/protobuf-format-better-tests
Better tests for protobuf format.
2021-02-24 15:47:39 +03:00
Maksim Kita
f274343cf0 Fixed Map type getExtremes 2021-02-22 23:32:43 +03:00
Alexey Milovidov
00e6b6232f Revert "optimize aggfunc column data copy (#19407)"
This reverts commit 7e3186c826.
2021-02-21 12:49:50 +03:00
Vitaly Baranov
2eecaee08d Better tests for protobuf format. 2021-02-20 23:13:32 +03:00
Alexey Milovidov
a9d9a6d56f Merge branch 'master' into in-memory-compression 2021-02-19 12:01:33 +03:00
Alexey Milovidov
04cb91a0fd Fix error 2021-02-18 06:02:41 +03:00
Alexey Milovidov
5007f7f018 Fix typo 2021-02-18 05:57:15 +03:00
Alexey Milovidov
634be2b933 Fix error 2021-02-18 03:52:09 +03:00
Alexey Milovidov
b7011f4f9c Fix build 2021-02-18 02:52:07 +03:00
Alexey Milovidov
1781a64370 Whitespaces 2021-02-18 02:11:41 +03:00
Alexey Milovidov
9b72255ca4 Implement compression for all columns except LowCardinality 2021-02-18 02:09:46 +03:00
Vitaly Baranov
18e036d19b Improved serialization for data types combined of Arrays and Tuples.
Improved matching enum data types to protobuf enum type.
Fixed serialization of the Map data type.
Omitted values are now set by default.
2021-02-17 20:50:09 +03:00
Alexey Milovidov
96dc69609c Fix Arcadia 2021-02-15 00:00:18 +03:00
Alexey Milovidov
34f5ea00e4 Merge branch 'master' into in-memory-compression 2021-02-13 02:08:40 +03:00
Alexey Milovidov
b5826121db Fix Arcadia 2021-02-13 00:29:06 +03:00
Maksim Kita
f9527738c9 Added comments 2021-02-12 12:12:04 +03:00
Alexey Milovidov
170daa5d65 Generate ya.make 2021-02-12 05:33:39 +03:00
Alexey Milovidov
71d84b9f67 Fix style 2021-02-12 03:52:53 +03:00
Alexey Milovidov
ed7270dd8b Better interface 2021-02-12 03:25:00 +03:00
alexey-milovidov
5a4a5fda20
Update ColumnVector.h 2021-02-12 01:04:55 +03:00
Maksim Kita
b276eac197 Common types template instantiations 2021-02-12 00:54:50 +03:00
Alexey Milovidov
5355175e49 Development 2021-02-12 00:26:14 +03:00
Alexey Milovidov
9b4f6fb711 Don't compress small blocks 2021-02-11 16:41:21 +03:00
Alexey Milovidov
d539948fe7 In memory compression: a prototype 2021-02-11 16:40:15 +03:00
templarzq
7e3186c826
optimize aggfunc column data copy (#19407)
* optimize agg column data copy

* add aggfunc col data copy test

* fix null warning

* fix style

* fix style

* fix style

* fix style

* fix copy same dat to same pos bug

* fix style

* fix style

* Update ColumnAggregateFunction.cpp

fix type check

* control record buf size

* fix performance

* fix performance

* fix destroy data speed

* decrease run time.

* fix warning

* decrease test run time

* Update aggfunc_col_data_copy.xml

* replace unordered_map with HashMap

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix hash map compile err

* fix compile err

* fix compile err

* fix format

* faster test

Co-authored-by: centos7 <templarzq@localhost.localdomain>
2021-02-08 16:18:48 +03:00
alesapin
011109c82a
Merge pull request #17348 from xjewer/alex/CLICKHOUSE-606_deduplication_UUID
CLICKHOUSE-606: query deduplication based on parts' UUID
2021-02-05 22:47:34 +03:00
Alexey Milovidov
12485eee6b Fix some of the issues found by Coverity 2021-02-02 22:07:23 +03:00
Aleksei Semiglazov
921518db0a CLICKHOUSE-606: query deduplication based on parts' UUID
* add the query data deduplication excluding duplicated parts in MergeTree family engines.

query deduplication is based on parts' UUID which should be enabled first with merge_tree setting
assign_part_uuids=1

allow_experimental_query_deduplication setting is to enable part deduplication, default ot false.

data part UUID is a mechanism of giving a data part a unique identifier.
Having UUID and deduplication mechanism provides a potential of moving parts
between shards preserving data consistency on a read path:
duplicated UUIDs will cause root executor to retry query against on of the replica explicitly
asking to exclude encountered duplicated fingerprints during a distributed query execution.

NOTE: this implementation don't provide any knobs to lock part and hence its UUID. Any mutations/merge will
update part's UUID.

* add _part_uuid virtual column, allowing to use UUIDs in predicates.

Signed-off-by: Aleksei Semiglazov <asemiglazov@cloudflare.com>

address comments
2021-02-02 16:53:39 +00:00
Anton Popov
031132038b fix filtering by uint8 greater than 127 2021-01-29 10:39:18 +03:00
Alexey Milovidov
5f13fdddc0 Remove useless code from BigInt 2021-01-26 21:22:40 +03:00
alexey-milovidov
78fff6bc39
Merge branch 'master' into multiple-nested 2021-01-15 14:54:27 +03:00
Alexey Milovidov
522ccddbec Better code 2021-01-14 23:05:00 +03:00
Alexey Milovidov
f60c5f8296 Merge branch 'master' into templarzq/master 2021-01-14 22:33:11 +03:00
templarzq
8b98b467b6
Update ColumnAggregateFunction.cpp 2021-01-13 13:57:42 +08:00
templarzq
c310f69dc3
Update ColumnAggregateFunction.cpp 2021-01-13 10:35:33 +08:00
templarzq
36381ce35e
Update ColumnAggregateFunction.cpp 2021-01-12 14:36:26 +08:00
Anton Popov
36ae0e4d35 Merge remote-tracking branch 'upstream/master' into HEAD 2021-01-11 13:51:12 +03:00
templarzq
8704136300
Update ColumnAggregateFunction.cpp 2021-01-11 14:38:21 +08:00
templarzq
7d1842cb19
Update ColumnAggregateFunction.cpp 2021-01-11 10:49:18 +08:00
templarzq
bf90fcc98e
Update ColumnAggregateFunction.cpp
fix warning
2021-01-11 10:40:03 +08:00
templarzq
b57c353289
Update ColumnAggregateFunction.h 2021-01-08 16:23:33 +08:00
templarzq
f32f6a0863
Update ColumnAggregateFunction.h 2021-01-08 16:10:36 +08:00
templarzq
ee8e3f94e7
Update ColumnAggregateFunction.cpp
convert eol to LF
2021-01-08 16:08:12 +08:00
templarzq
c7a3ae305a
Update ColumnAggregateFunction.cpp
fix style check
2021-01-08 15:14:42 +08:00
templarzq
ab9a3eaecb
Update ColumnAggregateFunction.cpp 2021-01-08 14:21:44 +08:00
templarzq
1769a51f70
Update ColumnAggregateFunction.h 2021-01-07 15:34:32 +08:00
templarzq
0b6c0d474f
Update ColumnAggregateFunction.cpp 2021-01-07 15:33:42 +08:00
Alexey Milovidov
25b100761f Merge branch 'master' into function_byte_size 2021-01-03 04:36:48 +03:00
Alexey Milovidov
a368a6802d Move implementation inside out 2021-01-03 01:58:10 +03:00
Alexey Milovidov
89595e184f Consistency 2021-01-02 00:26:41 +03:00